Xiaolin Tang [Wed, 23 Nov 2022 03:44:56 +0000 (11:44 +0800)]
Use GCC builtins for lrint functions if desired.
This patch is using the corresponding GCC builtin for lrintf, lrint,
lrintl and lrintf128 if the USE_FUNCTION_BUILTIN macros are defined to one
in math-use-builtins-function.h.
i386: Avoid rely on linker optimization to avoid relocation
lld does not implement all the linker optimization to avoid the GOT
relocation as done by binutils (bfd/elf32-i386.c:elf_i386_convert_load_reloc).
The current 'movl main@GOT(%ebx), %eax' will then create a GOT
relocation when building with lld, which make static-pie status to
not being able to start the provided main function.
The change uses a __wrap_main local symbol, which in turn calls main
(similar as used by aarch64 and s390x).
Checked on i686-linux-gnu with binutils and lld. Reviewed-by: Fangrui Song <maskray@google.com>
Florian Weimer [Tue, 8 Nov 2022 13:15:02 +0000 (14:15 +0100)]
Linux: Support __IPC_64 in sysvctl *ctl command arguments (bug 29771)
Old applications pass __IPC_64 as part of the command argument because
old glibc did not check for unknown commands, and passed through the
arguments directly to the kernel, without adding __IPC_64.
Applications need to continue doing that for old glibc compatibility,
so this commit enables this approach in current glibc.
For msgctl and shmctl, if no translation is required, make
direct system calls, as we did before the time64 changes. If
translation is required, mask __IPC_64 from the command argument.
For semctl, the union-in-vararg argument handling means that
translation is needed on all architectures.
Zong Li [Wed, 9 Nov 2022 14:40:59 +0000 (11:40 -0300)]
riscv: Get level 3 cache's information
RISC-V architecture extends the cache information for level 3 cache
in AUX vector in Linux v.6.1-rc1. This patch supports sysconf to get
the level 3 cache information.
наб [Wed, 9 Nov 2022 12:48:46 +0000 (13:48 +0100)]
iconvdata/tst-table-charmap.sh: remove handling of old, borrowed format
This "Old POSIX/DKUUG borrowed format" handling is original to the file
and doesn't seem to have ever been used, i.e. id/t-t-c doesn't seem to
have ever been called with argv[1] == POSIX.
Upcoming is a POSIX charmap, which would inadvertently trigger this.
Fangrui Song [Wed, 9 Nov 2022 04:56:59 +0000 (20:56 -0800)]
Makerules: Generate shlib.lds with -fuse-ld=bfd
lld does not dump a linker script with --verbose (it does not use a
linker script driven design and lots of linker processing is not
serializable as a linker script anyway). With the default
--with-default-link=no build, $@T is empty and makes `test -s $@T` fail.
Just dump the linker script with -fuse-ld=bfd. lld since 15
(https://reviews.llvm.org/D124656) supports custom RELRO sections in the
GNU ld dumped linker script.
Performance Changes:
Times are from N = 10 runs of the benchmark suite and are reported
as geometric mean of all ratios of New Implementation / Best Old
Implementation. Best Old Implementation was determined with the
highest ISA implementation.
Code Size Changes:
This change increase the size of libc.so by ~5.5kb bytes. For
reference the patch optimizing the normal strcpy family functions
decreases libc.so by ~5.2kb.
Full check passes on x86-64 and build succeeds for all ISA levels w/
and w/o multiarch.
Performance Changes:
Times are from N = 10 runs of the benchmark suite and are reported
as geometric mean of all ratios of New Implementation / Best Old
Implementation. Best Old Implementation was determined with the
highest ISA implementation.
Code Size Changes:
This change increase the size of libc.so by ~6.3kb bytes. For
reference the patch optimizing the normal strcpy family functions
decreases libc.so by ~5.7kb.
Full check passes on x86-64 and build succeeds for all ISA levels w/
and w/o multiarch.
Noah Goldstein [Wed, 9 Nov 2022 01:38:39 +0000 (17:38 -0800)]
x86: Optimize and shrink st{r|p}{n}{cat|cpy}-avx2 functions
Optimizations are:
1. Use more overlapping stores to avoid branches.
2. Reduce how unrolled the aligning copies are (this is more of a
code-size save, its a negative for some sizes in terms of
perf).
3. For st{r|p}n{cat|cpy} re-order the branches to minimize the
number that are taken.
Performance Changes:
Times are from N = 10 runs of the benchmark suite and are
reported as geometric mean of all ratios of
New Implementation / Old Implementation.
Noah Goldstein [Wed, 9 Nov 2022 01:38:38 +0000 (17:38 -0800)]
x86: Optimize and shrink st{r|p}{n}{cat|cpy}-evex functions
Optimizations are:
1. Use more overlapping stores to avoid branches.
2. Reduce how unrolled the aligning copies are (this is more of a
code-size save, its a negative for some sizes in terms of
perf).
3. Improve the loop a bit (similiar to what we do in strlen with
2x vpminu + kortest instead of 3x vpminu + kmov + test).
4. For st{r|p}n{cat|cpy} re-order the branches to minimize the
number that are taken.
Performance Changes:
Times are from N = 10 runs of the benchmark suite and are
reported as geometric mean of all ratios of
New Implementation / Old Implementation.
I couldn't find a way to merge them without making the
ifdefs incredibly difficult to follow.
2. All implementations can be made evex512 by including
"x86-evex512-vecs.h" at the top.
3. All implementations have an optional define:
`USE_EVEX_MASKED_STORE`
Setting to one uses evex-masked stores for handling short
strings. This saves code size and branches. It's disabled
for all implementations are the moment as there are some
serious drawbacks to masked stores in certain cases, but
that may be fixed on future architectures.
Full check passes on x86-64 and build succeeds for all ISA levels w/
and w/o multiarch.
Noah Goldstein [Sat, 29 Oct 2022 20:19:59 +0000 (15:19 -0500)]
x86: Use VMM API in memcmpeq-evex.S and minor changes
Changes to generated code are:
1. In a few places use `vpcmpeqb` instead of `vpcmpneq` to save a
byte of code size.
2. Add a branch for length <= (VEC_SIZE * 6) as opposed to doing
the entire block of [VEC_SIZE * 4 + 1, VEC_SIZE * 8] in a
single basic-block (the space to add the extra branch without
changing code size is bought with the above change).
Change (2) has roughly a 20-25% speedup for sizes in [VEC_SIZE * 4 +
1, VEC_SIZE * 6] and negligible to no-cost for [VEC_SIZE * 6 + 1,
VEC_SIZE * 8]
Linux: Add ppoll fortify symbol for 64 bit time_t (BZ# 29746)
Similar to ppoll, the poll.h header needs to redirect the poll call
to a proper fortified ppoll with 64 bit time_t support.
The implementation is straightforward, just need to add a similar
check as __poll_chk and call the 64 bit time_t ppoll version. The
debug fortify tests are also extended to cover 64 bit time_t for
affected ABIs.
Unfortunately it requires an aditional symbol, which makes backport
tricky. One possibility is to add a static inline version if compiler
supports is and call abort instead of __chk_fail, so fortified version
will call __poll64 in the end.
Another possibility is to just remove the fortify support for
_TIME_BITS=64.
For clang the redeclaration after the first use, the visibility attribute
is silently ignored (symbol is STV_DEFAULT) while the asm label attribute
causes an error.
With clang if the main file is <stdin>, the first non-main-file
dependency is not listed in the -MP output. Although it was fixed
on clang-16 [1], this change adds portability for older version.
Florian Weimer [Fri, 4 Nov 2022 06:43:59 +0000 (07:43 +0100)]
posix: Make posix_spawn extensions available by default
Some sources merely include <spawn.h> without -D_GNU_SOURCE and expect
declarations for posix_spawn_file_actions_addchdir_np to be available.
For consistency, declare posix_spawn_file_actions_addfchdir_np,
posix_spawn_file_actions_addclosefrom_np,
posix_spawn_file_actions_addtcsetpgrp_np as well.
Sunil K Pandey [Tue, 9 Aug 2022 14:57:29 +0000 (07:57 -0700)]
x86_64: Implement evex512 version of strrchr and wcsrchr
Changes from v1:
Use vec api for register.
Replace VPCMP with VPCMPEQ
Restructure and remove 1 unconditional jump.
Change page cross logic to use sall.
This patch implements following evex512 version of string functions.
evex512 version takes up to 30% less cycle as compared to evex,
depending on length and alignment.
- strrchr function using 512 bit vectors.
- wcsrchr function using 512 bit vectors.
Florian Weimer [Thu, 3 Nov 2022 16:28:03 +0000 (17:28 +0100)]
elf: Introduce <dl-call_tls_init_tp.h> and call_tls_init_tp (bug 29249)
This makes it more likely that the compiler can compute the strlen
argument in _startup_fatal at compile time, which is required to
avoid a dependency on strlen this early during process startup.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Without this change, parse failures result in an exception:
Traceback (most recent call last):
File "tst-glibcelf.py", line 23, in <module>
import glibcelf
File "/path/to/git/scripts/glibcelf.py", line 226, in <module>
_elf_h = _parse_elf_h()
File "/path/to/git/scripts/glibcelf.py", line 221, in _parse_elf_h
result = glibcpp.macro_eval(glibcpp.macro_definitions(tokens), reporter)
File "/path/to/git/scripts/glibcpp.py", line 379, in macro_eval
reporter.error(md.line, 'macro {} redefined'.format(md.name))
File "/path/to/git/scripts/glibcelf.py", line 214, in error
errors += 1
UnboundLocalError: local variable 'errors' referenced before assignment
Florian Weimer [Thu, 3 Nov 2022 08:39:31 +0000 (09:39 +0100)]
elf: Rework exception handling in the dynamic loader [BZ #25486]
The old exception handling implementation used function interposition
to replace the dynamic loader implementation (no TLS support) with the
libc implementation (TLS support). This results in problems if the
link order between the dynamic loader and libc is reversed (bug 25486).
The new implementation moves the entire implementation of the
exception handling functions back into the dynamic loader, using
THREAD_GETMEM and THREAD_SETMEM for thread-local data support.
These depends on Hurd support for these macros, added in commit b65a82e4e757c1e6cb7073916 ("hurd: Add THREAD_GET/SETMEM/_NC").
One small obstacle is that the exception handling facilities are used
before the TCB has been set up, so a check is needed if the TCB is
available. If not, a regular global variable is used to store the
exception handling information.
Also rename dl-error.c to dl-catch.c, to avoid confusion with the
dlerror function.
Szabolcs Nagy [Tue, 11 Oct 2022 14:24:41 +0000 (15:24 +0100)]
Fix OOB read in stdlib thousand grouping parsing [BZ #29727]
__correctly_grouped_prefixmb only worked with thousands_len == 1,
otherwise it read past the end of cp or thousands.
This affects scanf formats like %'d, %'f and the internal but
exposed __strto{l,ul,f,d,..}_internal with grouping flag set
and an LC_NUMERIC locale where thousands_len > 1.
Avoid OOB access by considering thousands_len when initializing cp.
This fixes bug 29727.
Found by the morello port with strict bounds checking where
Aurelien Jarno [Tue, 1 Nov 2022 19:43:55 +0000 (20:43 +0100)]
linux: Fix fstatat on MIPSn64 (BZ #29730)
Commit 6e8a0aac2f883 ("time: Fix overflow itimer tests on 32-bit
systems") changed in_time_t_range to assume a 32-bit time_t. This broke
fstatat on MIPSn64 that was using it with a 64-bit time_t due to
difference between stat and stat64. This commit fix that by adding a
MIPSn64 specific version, which bypasses the EOVERFLOW tests.
The maximum number of directives is already limited by the maximum
value of iovec, and current padding usage on _dl_map_object_from_fd
specifies a value of 16 (2 times sizeof (void *)) in hexa, which is
less than the INT_STRLEN_BOUND(void *) (20 for LP64).
This works if pointers are larger than 8 bytes, for instance 16.
In this case the maximum padding would be 32 and the IFMTSIZE would
be 40.
The resulting code does use a slightly larger static stack, the
output of -fstack-usage (for x86_64):
clang complains that libc_hidden_data_def (__nptl_threads_events)
creates an invalid alias:
pthread_create.c:50:1: error: alias must point to a defined variable or function
libc_hidden_data_def (__nptl_threads_events)
^
../include/libc-symbols.h:621:37: note: expanded from macro
'libc_hidden_data_def'
It seems that clang requires that a proper prototype is defined prior
the hidden alias creation.
configure: Use -Wno-ignored-attributes if compiler warns about multiple aliases
clang emits an warning when a double alias redirection is used, to warn
the the original symbol will be used even when weak definition is
overridden. However, this is a common pattern for weak_alias, where
multiple alias are set to same symbol.
intl: Fix clang -Wunused-but-set-variable on plural.c
Clang warns that '__gettextnerrs' set but not used:
intl/plural.c:1034:9: error: variable '__gettextnerrs' set but not used
[-Werror,-Wunused-but-set-variable]
int yynerrs = 0;
^
Clang 15 (https://reviews.llvm.org/D122271) -Wunused-but-set-variable
gives a warning while GCC doesn't. The -Wunused-but-set-variable is
available in GCC 4.6, lower than the minimum required version 6.2.
Since the file is auto-generated, suppress the warning with a compiler
flag.
Joseph Myers [Mon, 31 Oct 2022 23:20:08 +0000 (23:20 +0000)]
Fix build with GCC 13 _FloatN, _FloatNx built-in functions
GCC 13 has added more _FloatN and _FloatNx versions of existing
<math.h> and <complex.h> built-in functions, for use in libstdc++-v3.
This breaks the glibc build because of how those functions are defined
as aliases to functions with the same ABI but different types. Add
appropriate -fno-builtin-* options for compiling relevant files, as
already done for the case of long double functions aliasing double
ones and based on the list of files used there.
I fixed some mistakes in that list of double files that I noticed
while implementing this fix, but there may well be more such
(harmless) cases, in this list or the new one (files that don't
actually exist or don't define the named functions as aliases so don't
need the options). I did try to exclude cases where glibc doesn't
define certain functions for _FloatN or _FloatNx types at all from the
new uses of -fno-builtin-* options. As with the options for double
files (see the commit message for commit 49348beafe9ba150c9bd48595b3f372299bddbb0, "Fix build with GCC 10 when
long double = double."), it's deliberate that the options are used
even if GCC currently doesn't have a built-in version of a given
functions, so providing some level of future-proofing against more
such built-in functions being added in future.
Tested with build-many-glibcs.py for aarch64-linux-gnu
powerpc-linux-gnu powerpc64le-linux-gnu x86_64-linux-gnu (compilers
and glibcs builds) with GCC mainline.
H.J. Lu [Fri, 28 Oct 2022 22:16:32 +0000 (15:16 -0700)]
elf: Build tst-relr-mod[34]a.so with $(LDFLAGS-rpath-ORIGIN)
When --enable-hardcoded-path-in-tests is used only with DT_RUNPATH,
elf/tst-relr3 and elf/tst-relr4 failed to run. Their dependency
libraries, tst-relr-mod3a.so and tst-relr-mod4a.so, are failed to
load since DT_RUNPATH on executable doesn't apply to them. Build
tst-relr-mod3a.so and tst-relr-mod4a.so with $(LDFLAGS-rpath-ORIGIN)
to add DT_RUNPATH for their dependency libraries. Reviewed-by: Fangrui Song <maskray@google.com>
Sunil K Pandey [Mon, 3 Oct 2022 19:00:53 +0000 (12:00 -0700)]
x86-64: Improve evex512 version of strlen functions
This patch improves following functionality
- Replace VPCMP with VPCMPEQ.
- Replace page cross check logic with sall.
- Remove extra lea from align_more.
- Remove uncondition loop jump.
- Use bsf to check max length in first vector.
Correctly determine libc.so 'OUTPUT_FORMAT' when cross-compiling.
Commit 87d583c6e8cd0e49f64da76636ebeec033298b4d replaces the sed script
with an "objdump -f" invocation to determine the 'OUTPUT_FORMAT' bit of
the libc.so linker script.
However, when cross-compiling, for example from x86_64-linux-gnu to
aarch64-linux-gnu, "objdump -f" would report the wrong
format ("elf64-little"). Conversely, "aarch64-linux-gnu-objdump -f"
reports "elf64-littleaarch64" as expected.
This patch changes 'configure.ac' to use AC_CHECK_TOOL rather than
'$CC -print-prog-name=objdump' to determine the value of the OBJDUMP
variable. That way, OBJDUMP is set to TRIPLET-objdump when
cross-compiling for TRIPLET. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
Letu Ren [Fri, 21 Oct 2022 14:54:50 +0000 (22:54 +0800)]
stdlib/strfrom: Add copysign to fix NAN issue on riscv (BZ #29501)
According to the specification of ISO/IEC TS 18661-1:2014,
The strfromd, strfromf, and strfroml functions are equivalent to
snprintf(s, n, format, fp) (7.21.6.5), except the format string contains only
the character %, an optional precision that does not contain an asterisk *, and
one of the conversion specifiers a, A, e, E, f, F, g, or G, which applies to
the type (double, float, or long double) indicated by the function suffix
(rather than by a length modifier). Use of these functions with any other 20
format string results in undefined behavior.
strfromf will convert the arguement with type float to double first.
According to the latest version of IEEE754 which is published in 2019,
Conversion of a quiet NaN from a narrower format to a wider format in the same
radix, and then back to the same narrower format, should not change the quiet
NaN payload in any way except to make it canonical.
When either an input or result is a NaN, this standard does not interpret the
sign of a NaN. However, operations on bit strings—copy, negate, abs,
copySign—specify the sign bit of a NaN result, sometimes based upon the sign
bit of a NaN operand. The logical predicates totalOrder and isSignMinus are
also affected by the sign bit of a NaN operand. For all other operations, this
standard does not specify the sign bit of a NaN result, even when there is only
one input NaN, or when the NaN is produced from an invalid operation.
converting NAN or -NAN with type float to double doesn't need to keep
the signbit. As a result, this test case isn't mandatory.
The problem is that according to RISC-V ISA manual in chapter 11.3 of
riscv-isa-20191213,
Except when otherwise stated, if the result of a floating-point operation is
NaN, it is the canonical NaN. The canonical NaN has a positive sign and all
significand bits clear except the MSB, a.k.a. the quiet bit. For
single-precision floating-point, this corresponds to the pattern 0x7fc00000.
which means that conversion -NAN from float to double won't keep the signbit.
Since glibc ought to be consistent here between types and architectures, this
patch adds copysign to fix this problem if the string is NAN. This patch
adds two different functions under sysdeps directory to work around the
issue.
This patch has been tested on x86_64 and riscv64.
Resolves: BZ #29501
v2: Change from macros to different inline functions.
v3: Add unlikely check to isnan.
v4: Fix wrong commit message header.
v5: Fix style: add space before parentheses.
v6: Add copyright. Signed-off-by: Letu Ren <fantasquex@gmail.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Szabolcs Nagy [Tue, 11 Oct 2022 13:57:16 +0000 (14:57 +0100)]
Fix missing NUL terminator in stdio-common/scanf13 test
sscanf is only defined on nul terminated string input, but '\0' was
missing in this test which caused _IO_str_init_static_internal to
read OOB on the stack when computing the bounds of the string.
Szabolcs Nagy [Tue, 11 Oct 2022 13:22:35 +0000 (14:22 +0100)]
elf: Fix alloca size in _dl_debug_vdprintf
The alloca size did not consider the optional width parameter for
padding which could cause buffer underflow. The width is currently used
e.g. by _dl_map_object_from_fd which passes 2 * sizeof(void *) which
can be larger than the alloca buffer size on targets where
sizeof(void *) >= 2 * sizeof(unsigned long).
Even if large width is not used on existing targets it is better to fix
the formatting code to avoid surprises.
Szabolcs Nagy [Fri, 29 Jul 2022 07:00:06 +0000 (08:00 +0100)]
aarch64: Fix the extension header write in getcontext and swapcontext
The extension header is two 32bit words and in the last header both
should be 0. There is plenty space in the __reserved area, but it's
better not to write more than we mean to.
Florian Weimer [Thu, 27 Oct 2022 09:36:44 +0000 (11:36 +0200)]
elf: Introduce to _dl_call_fini
This consolidates the destructor invocations from _dl_fini and
dlclose. Remove the micro-optimization that avoids
calling _dl_call_fini if they are no destructors (as dlclose is quite
expensive anyway). The debug log message is now printed
unconditionally.
Unexpected output from check-localplt: …/elf/ld.so.jmprel:
*** DT_JMPREL does not match any section's address
This commit changes the script to record the DT_PLTRELSZ value and
reject DT_JMPREL values not a section boundary only if DT_PLTRELSZ
is present with a non-zero value.
Wilco Dijkstra [Wed, 26 Oct 2022 13:16:50 +0000 (14:16 +0100)]
aarch64: Use memcpy_simd as the default memcpy
Since __memcpy_simd is the fastest memcpy on almost all cores, replace
the generic memcpy with it. If SVE is available, a SVE memcpy will be
used by default (including for Neoverse N2).
x86_64: Implement evex512 version of strchrnul, strchr and wcschr
This patch implements following evex512 version of string functions.
evex512 version takes up to 30% less cycle as compared to evex,
depending on length and alignment.
- strchrnul function using 512 bit vectors.
- strchr function using 512 bit vectors.
- wcschr function using 512 bit vectors.
linux: Fix generic struct_stat for 64 bit time (BZ# 29657)
The generic Linux struct_stat misses the conditionals to use
bits/struct_stat_time64_helper.h in the __USE_TIME_BITS64 for
architecture that uses __TIMESIZE == 32 (currently csky and nios2).
Since newer ports should not support 32 bit time_t, the generic
implementation should be used as default.
For arm, hppa, and sh a copy of default struct_stat is added,
while for csky and nios a new one based on generic is used, along
with conditionals to use bits/struct_stat_time64_helper.h.
The default struct_stat is also replaced with the generic one.
Checked on aarch64-linux-gnu and arm-linux-gnueabihf.
Joseph Myers [Mon, 24 Oct 2022 13:43:19 +0000 (13:43 +0000)]
Add ADDRB from Linux 6.0 to bits/termios-c_cflag.h
Linux 6.0 adds a constant ADDRB, a termios c_cflag bit, to its
include/uapi/asm-generic/termbits-common.h.
Add it accordingly to glibc's bits/termios-c_cflag.h headers. As
other constants in these headers are generally in octal, I converted
the value to octal to match. As ADDRB isn't in a POSIX-reserved
namespace, I made it conditional on __USE_MISC.
Noah Goldstein [Thu, 20 Oct 2022 02:15:55 +0000 (19:15 -0700)]
x86: Add support for VEC_SIZE == 64 in strcmp-evex.S impl
Unused at the moment, but evex512 strcmp, strncmp, strcasecmp{l}, and
strncasecmp{l} functions can be added by including strcmp-evex.S with
"x86-evex512-vecs.h" defined.
x86: Optimize strrchr-evex.S and implement with VMM headers
Added `vpcompress{b|d}` to the page-cross logic with is an
AVX512-VBMI2 instruction. This is not supported on SKX. Since the
page-cross logic is relatively cold and the benefit is minimal
revert the page-cross case back to the old logic which is supported
on SKX.
Felix Riemann [Tue, 18 Oct 2022 15:20:44 +0000 (17:20 +0200)]
sysdeps: arm: Fix preconfigure script for ARMv8/v9 targets [BZ #29698]
The ARM preconfigure script tries to detect the capabilities of the
target platform by checking the compiler's predefined architecture
macros. However, if the compiler is tuning for AArch32 on ARMv8/v9 this
step fails:
checking for sysdeps preconfigure fragments... aarch64 alpha arc arm
WARNING: arm/preconfigure: Did not find ARM architecture type; using default
This is because preconfigure.ac doesn't escape the square brackets in
the glob for matching compilers targeting ARMv8. Adding another pair of
brackets to escape the first pair fixes this:
checking for sysdeps preconfigure fragments... aarch64 alpha arc arm
Found compiler is configured for something newer than v7 - using v7
Signed-off-by: Felix Riemann <felix.riemann@sma.de> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
projects / thirdparty / glibc.git / log
