rootvector2 [Thu, 28 May 2026 06:59:23 +0000 (12:29 +0530)]
xz: Fix --files/--files0 usage via XZ_OPT and XZ_DEFAULTS env vars
If --files=FILELIST or --files0=FILELIST was specified via an enrivonment
variable, it worked if there were no errors when reading from the file
FILELIST. However, if an error occurred when reading from FILELIST,
there was a use-after-free bug when printing the error message because
the memory containing the string "FILELIST" had been freed.
Lasse Collin [Wed, 20 May 2026 18:21:51 +0000 (21:21 +0300)]
xz: Reduce indentation of three tables on the man page
The extra indentation makes the tables stand out from the text, and
in the English man page it works well. However, some translations
have strings that require more space, and the table might become
too wide. In some translations this has been solved by splitting
the translated string on two lines, for example, Romanian translation
of "4096-byte alignment is best".
On Debian 13 with groff 1.23.0, the "man" command shows a few warnings:
$ LANGUAGE=it MANWIDTH=80 man --warnings xz > /dev/null
<standard input>:477: warning: table wider than line length minus indentation
<standard input>:545: warning: table wider than line length minus indentation
<standard input>:1182: warning: table wider than line length minus indentation
German man page shows those warnings too. On Arch with groff 1.24.1,
there are fewer warnings because the newer groff indents less, and
fewer tables reach or exceed the right margin. Thus, some safety margin
is good to keep the output neat with different man page formatters.
It feels silly to reduce the indentation on the English man page which
has plenty of safety margin, but it's simpler than trying to do it
only for the translated man pages. This doesn't affect readability;
it's just cosmetic.
I tried using .IP instead of .RS + .PP for the tables, but on Solaris 10
it results in an extra empty line before the table. It makes some
difference on AIX 7.3 too. With groff, both are fine. Stick to
.RS + .PP + .TS ... .TE + .RE for slightly better portability.
Co-authored-by: Otto Kekäläinen <otto@debian.org>
Partially-fixes: https://github.com/tukaani-project/xz/pull/220
troff:man/ko/xz.1:1405: warning: color 'i' not defined
troff:man/ko/xzdec.1:17: warning: cannot select font '\'
troff:man/ko/xzdec.1:19: warning: cannot select font '\'
Lasse Collin [Wed, 20 May 2026 10:09:59 +0000 (13:09 +0300)]
Man pages: Improve rendering in mandoc
Don't use "" as an argument to .IP because it makes mandoc insert
an unwanted extra empty line. With the xz man page, this change makes
no difference in plain text output with GNU groff or on AIX or
Solaris 10. The list in the xzgrep man page has a different indentation
now because previously it had two arguments (.IP "" 4), but that's OK.
I don't remember why '.IP' was changed to '.IP ""' in cec0ddc8ec4c.
Maybe it made a difference on some proprietary UNIX over 15 years ago
or maybe it was just a misunderstanding.
Lasse Collin [Tue, 19 May 2026 22:25:41 +0000 (01:25 +0300)]
po4a/update-po: Workaround an issue with non-ASCII chars in tables
Non-ASCII chars in tables (.TS ... .TE) result in warnings:
man --warnings po4a/man/uk/xz.1 > /dev/null
troff:<standard input>:464: warning: special character 'u0411\[tbl' not defined
troff:<standard input>:1157: warning: special character 'u044\[tbl' not defined
troff:<standard input>:1682: warning: special character 'u0434\[tbl' not defined
po4a generates groff Unicode escapes like '\u0435' followed immediately
by '[' in table contexts, causing groff to interpret the full string
'u0435[tbl' as a single special character name. Fix this by inserting
a dummy character (\&) after every Unicode escape so that the bracket
is treated as literal text instead of part of the character name.
Co-authored-by: Otto Kekäläinen <otto@debian.org>
Partially-fixes: https://github.com/tukaani-project/xz/pull/220
Lasse Collin [Tue, 19 May 2026 14:36:15 +0000 (17:36 +0300)]
liblzma: ARM64 & LoongArch: Avoid aligned_readXXle on unaligned pointers
While most ARM64 and LoongArch processors support unaligned access,
the code wouldn't have worked on strict-align processors.
Use byte-by-byte method if the input buffer is at most 7 bytes. This
way the code works on strict-align processors too. It might not be
the very best method for tiny buffers, but it shouldn't matter in
practice (the simple implementation for big buffers isn't amazing
either because it cannot execute more than one CRC32 instruction
at a time on superscalar processors).
Reported-by: Guanni Qu Fixes: 0ed893668554 ("liblzma: ARM64 CRC32: Align the buffer faster") Fixes: 7baf6835cfbf ("liblzma: Speed up CRC32 calculation on 64-bit LoongArch")
lzma_stream_buffer_decode() would often fail with LZMA_BUF_ERROR
when it should have failed with LZMA_DATA_ERROR.
In debug builds this also resulted in an assertion failure.
In the _buffer_decode() functions, LZMA_BUF_ERROR is only used when
the output buffer is too small. Truncated input is LZMA_DATA_ERROR.
Reported-by: Guanni Qu Reported-by: Omkhar Arasaratnam Fixes: c81f13ff2927 ("Added lzma_stream_buffer_decode() and made minor cleanups.")
Lasse Collin [Tue, 19 May 2026 09:34:53 +0000 (12:34 +0300)]
xz: Fix use-after-free in verbose error message to a non-tty
If --verbose was used when stderr isn't a terminal, the filename
contained a non-printable character, and an error occurred,
a use-after-free occurred when printing the error message.
The bug exists in versions 5.7.1alpha to 5.8.3.
Reported-by: Omkhar Arasaratnam Fixes: d22f96921fd2 ("xz: Use tuklib_mbstr_nonprint")
Lasse Collin [Sun, 29 Mar 2026 16:11:21 +0000 (19:11 +0300)]
liblzma: Fix a buffer overflow in lzma_index_append()
If lzma_index_decoder() was used to decode an Index that contained no
Records, the resulting lzma_index had an invalid internal "prealloc"
value. If lzma_index_append() was called on this lzma_index, too
little memory would be allocated and a buffer overflow would occur.
While this combination of the API functions is meant to work, in the
real-world apps this call sequence is rare or might not exist at all.
This bug is older than xz 5.0.0, so all stable releases are affected.
Lasse Collin [Fri, 27 Mar 2026 19:15:12 +0000 (21:15 +0200)]
CMake: Adjust warnings with MSVC and clang-cl
When using clang-cl, avoid the long list of warning options used with
GCC and Clang because, for MSVC compatibility, clang-cl's -Wall behaves
like -Weverything.
Add warning options for MSVC. Use them with clang-cl too.
Lasse Collin [Wed, 25 Mar 2026 17:03:00 +0000 (19:03 +0200)]
xz: Prevent an integer overflow in --files and --files0
This requires a filename (or something that pretends to be a filename)
of at least 2 GiB on a 32-bit platform, and that realloc() to
SIZE_MAX / 2 + 1 bytes has succeeded.
Collin Funk [Mon, 16 Mar 2026 06:24:59 +0000 (23:24 -0700)]
xz: Don't fsync directories on QNX
On QNX calling fsync on a directory fails with EINVAL. This causes the
following test failure:
$ cat tests/test_suffix.sh.log
/data/home/root/xz-5.8.2/src/xz/.libs/lt-xz: suffix_temp: Synchronizing the directory of the file failed: Invalid argument
Failed to decompress a file with a suffix set in raw format
FAIL test_suffix.sh (exit status: 1)
Lasse Collin [Tue, 24 Feb 2026 16:22:25 +0000 (18:22 +0200)]
CMake: Fix a wrong #define
HAVE_SYS_PARAM_H was misspelled HAVE_PARAM_H. This might have broken
the build on systems where sysctl() is used to detect the amount of RAM
or the number of processor cores/threads.
The cpuset code for FreeBSD doesn't need the macro it so it was removed.
Radek Zikmund [Thu, 12 Feb 2026 14:24:15 +0000 (15:24 +0100)]
liblzma/MSVC: Add a cast to fix a warning (C4057) with __cpuid()
__cpuid from MSVC (and also the old Intel ICC) use int[], while
__get_cpuid from <cpuid.h> in GCC and Clang use unsigned int[].
Adding a cast is the simplest fix.
Radek Zikmund [Mon, 23 Feb 2026 16:50:00 +0000 (17:50 +0100)]
CMake: Guard script configure_file() calls with ENABLE_SCRIPTS
The configure_file() calls for xzdiff, xzgrep, xzmore, and xzless
were running unconditionally within the if(UNIX) block, even when
ENABLE_SCRIPTS was OFF. This would cause a build failure if the
src/scripts/*.in files were not present. Deleting those files can
simplify license compliance when the scripts aren't needed.
Move the foreach loop and related code inside if(ENABLE_SCRIPTS) guard
so that configure_file() is only called when scripts are actually needed.
This is mostly whitespace changes to adjust the indentation.
Co-authored-by: Lasse Collin <lasse.collin@tukaani.org> Closes: https://github.com/tukaani-project/xz/pull/210
Lasse Collin [Sun, 25 Jan 2026 11:30:46 +0000 (13:30 +0200)]
liblzma: Add the "License" property to liblzma.pc
The "License" property is a pkgconf extension. See pc(5).
pkg-config 0.29.2 (2017) ignores unrecognized properties so this
commit shouldn't create compatibility issues.
pkgconf provides bomtool which uses the "License" property to generate
a software bill of materials. In pkgconf.git there is spdxtool for the
same task but with different output format.
The pkgconf extensions "Copyright" and "Maintainer" would also be
used by bomtool and spdxtool, but I don't want to add those properties
at least for now.
Lasse Collin [Wed, 10 Dec 2025 17:11:27 +0000 (19:11 +0200)]
tuklib_integer/CMake: Don't run unaligned access checks unnecessarily
If the cache variable TUKLIB_FAST_UNALIGNED_ACCESS is already set,
the autodetection result isn't needed because the option() command
does nothing when the cache variable is already set.
This is largely white space change to indent the if...endif block.
Lasse Collin [Wed, 10 Dec 2025 14:49:55 +0000 (16:49 +0200)]
tuklib_integer: Use CMAKE_C_COMPILER_ARCHITECTURE_ID when available
CMake >= 4.1 sets CMAKE_<LANG>_COMPILER_ARCHITECTURE_ID on many
platforms. The list of possible values are documented. Use this
variable when available. On older CMake versions CMAKE_SYSTEM_PROCESSOR
is still used, thus the regexes have to include values like ^amd64 still.
With old CMake versions, checking CMAKE_C_COMPILER_ARCHITECTURE_ID
is somewhat useful with MSVC because CMAKE_SYSTEM_PROCESSOR might
not match the target architecture.
Lasse Collin [Wed, 10 Dec 2025 14:49:55 +0000 (16:49 +0200)]
tuklib_integer: Autodetect when -mstrict-align is used with GCC on ARM64
On ARM64, support for fast unaligned memory access was autodetected by
checking if __ARM_FEATURE_UNALIGNED is defined. However, at least GCC
versions up to 15.2.0 define the macro even when -mstrict-align has
been specified. Thus, autodetection with GCC doesn't work correctly,
and binaries built using -mstrict-align can be much slower than they
need to be, unless the user also passes --disable-unaligned-access
to configure or -DTUKLIB_FAST_UNALIGNED_ACCESS=OFF to cmake.
Lasse Collin [Tue, 9 Dec 2025 10:55:45 +0000 (12:55 +0200)]
xz: Never translate "Failed to enable the sandbox"
This was already the case in practice because I had forgotten to list
src/xz/sandbox.c in po/POTFILES.in. However, it seems better to never
translate this particular error message. It should almost never occur
and if it does, an untranslated message is should make it easier to
find bug reports about it.
Lasse Collin [Tue, 9 Dec 2025 10:13:36 +0000 (12:13 +0200)]
CMake: Autodetect unaligned access support on LoongArch
This still relies on CMAKE_SYSTEM_PROCESSOR. CMake 4.1 added more
CMAKE_<LANG>_COMPILER_ARCHITECTURE_ID values to detect the arch in
a more defined manner, but 4.1 is too new to require for now.
Thanks-to: Li Chenggang <lichenggang@deepin.org> Closes: https://github.com/tukaani-project/xz/pull/186
Lasse Collin [Tue, 9 Dec 2025 10:13:36 +0000 (12:13 +0200)]
Autotools: Autodetect unaligned access support on LoongArch
According to [1] sections 7.4, 8.1, and 8.2, desktop and server
processors support fast unaligned access, but embedded systems likely
don't.
It's important that TUKLIB_FAST_UNALIGNED_ACCESS isn't defined when
-mstrict-align is in use because it will result in slower binaries
even if running on a processor that supports fast unaligned access.
It's because compilers will translate multibyte memcpy() to multiple
byte-by-byte instructions instead of wider loads and stores. The
compression times from [2] show this well:
Unaligned access CFLAGS Compression time
enabled -O2 -mno-strict-align 66.1 s
disabled -O2 -mno-strict-align 79.5 s
disabled -O2 -mstrict-align 79.9 s
enabled -O2 -mstrict-align 129.1 s
There currently (GCC 15.2) is no preprocessor macro on LoongArch
to detect if -mstrict-align or -mno-strict-align is in effect (the
default is -mno-strict-align). Use heuristics to detect which of the
flags is in effect.
Lasse Collin [Sun, 2 Nov 2025 10:17:50 +0000 (12:17 +0200)]
liblzma: Silence two Coverity warnings
lzma_lzma_decoder_memusage() returns UINT64_MAX if lc/lp/pb aren't
valid. alone_decoder.c and lzip_decoder.c didn't check the return
value because in both it is known that lc/lp/pb are valid. Make them
call the _nocheck() variant instead which skips the validation (it
already existed for LZMA2's internal use).
Lasse Collin [Sun, 2 Nov 2025 10:17:50 +0000 (12:17 +0200)]
liblzma: Fix a harmless read of shared variable without mutex
The partial_update_mode enumeration had three states, _DISABLED,
_START, and _ENABLED. Main thread changed it from _DISABLED to _START
while holding a mutex. Once set to _START, worker thread changed it
to _ENABLED without a mutex. Later main thread read it without a mutex,
so it could see either _START or _ENABLED. However, it made no
difference because the main thread checked for != _DISABLED, so
it didn't matter if it saw _START or _ENABLED.
Nevertheless, such things must not be done. It's clear it was a mistake
because there were two comments that directly contradicted each
other about how the variable was accessed.
Split the enumeration into two booleans:
- partial_update_enabled: A worker thread locks the mutex to read
this variable and the main thread locks the mutex to change the
value. Because only the main thread modifies the variable, the
main thread can read the value without locking the mutex.
This variable replaces the _DISABLED -> _START transition.
- partial_update_started is for worker thread's internal use and thus
needs no mutex. This replaces the _START -> _ENABLED transition.
Fixes: Coverity CID 456025 Fixes: bd93b776c1bd ("liblzma: Fix a deadlock in threaded decoder.")
Lasse Collin [Tue, 25 Nov 2025 11:55:35 +0000 (13:55 +0200)]
Change the sorting order in THANKS
In short, sort the names with this command (-k1,1 isn't needed because
the lines with names start with " -"):
LC_ALL=en_US.UTF-8 sort -k2,2 -k3,3 -k4,4 -k5,5
When THANKS was created, I wrote the names as "First Last" and attempted
to keep them sorted by last name / surname / family name. This works
with many names in THANKS, but it becomes complicated with names that
don't fit that pattern. For example, names that are written as
"Last First" can be manually sorted by family name, but only if one
knows which part of the name is the family name.[*] And of course,
the concept of first/last name doesn't apply to all names.
[*] xz had a co-maintainer who could help me with such names,
but fortunately he isn't working on the project anymore.
Adding the names in chronological order could have worked too, although
if something is contributed by multiple people, one would still have to
decide how to sort the names within the batch. Another downside would
be that if THANKS is updated in more than one work-in-progress branch,
merge conflicts would occur more often.
Don't attempt to sort by last name. Let's be happy that people tend to
provide names that can be expressed in a reasonable number of printable
Unicode characters. In practice, people have been even nicer: if the
native language doesn't use a Latin script alphabet, people often provide
a transliterated name (only or in addition to the original spelling),
which is very much appreciated by those who don't know the native script.
Treat the names as opaque strings or space-separated strings for sorting
purposes. This means that most names will now be sorted by first name.
There still are many choices how to sort:
(1) LC_ALL=en_US.UTF-8 sort
The project is in English, so this may sound like a logical choice.
However, spaces have a lower weight than letters, which results in
this order:
- A Ba
- Ab C
- A Bc
- A Bd
(2) LC_ALL=en_US.UTF-8 sort -k2,2
This first sorts by the first word and then by the rest of the
string. It's -k2,2 instead of -k1,1 to skip the leading dash.
- A Ba
- A Bc
- A Bd
- Ab C
I like this more than (1). One could add -k3,3 -k4,4 -k5,5 ... too.
With current THANKS it makes no difference but it might some day.
NOTE: The ordering in en_US.UTF-8 can differ between libc versions
and operating systems. Luckily it's not a big deal in THANKS.
(3) LC_ALL=en_US.UTF-8 sort -f -k2,2
Passing -f (--ignore-case) to sort affects sorting of single-byte
characters but not multibyte characters (GNU coreutils 9.9):
No -f With -f LC_ALL=C
Aa A.A A.A
A.A Aa Aa
Ää Ää Ä.Ä
Ä.Ä Ä.Ä Ää
In GNU coreutils, the THANKS file is sorted using "sort -f -k1,1".
There is also a basic check that the en_US.UTF-8 locale is
behaving as expected.
(4) LC_ALL=C sort
This sorts by byte order which in UTF-8 is the same as Unicode
code point order. With the strings in (1) and (2), this produces
the same result as in (2). The difference in (3) can be seen above.
The results differ from en_US.UTF-8 when a name component starts
with a lower case ASCII letter (like "von" or "de"). Worse, any
non-ASCII characters sort after ASCII chars. These properties might
look weird in English language text, although it's good to remember
that en_US.UTF-8 sorting can appear weird too if one's native
language isn't English.
The choice between (2) and (4) was difficult but I went with (2).
Lasse Collin [Sun, 23 Nov 2025 18:13:49 +0000 (20:13 +0200)]
Landlock: Workaround a bug in RHEL 9 kernel
If one runs xz 5.8.0 or 5.8.1 from some other distribution in a container
on RHEL 9, xz will fail with the message "Failed to enable the sandbox".
RHEL 9 kernel since 5.14.0-603.el9 (2025-07-30) claims to support
Landlock ABI version 6, but it lacks support for LANDLOCK_SCOPE_SIGNAL.
The issue is still present in 5.14.0-643.el9 (2025-11-22). Red Hat is
aware of the issue, but I don't know when it will be fixed.
The sandbox is meant to be transparent to users, thus there isn't and
won't be a command line option to disable it. Instead, add a workaround
to keep xz working on the buggy RHEL 9 kernels.
Lasse Collin [Mon, 3 Nov 2025 10:41:31 +0000 (12:41 +0200)]
mythread.h: Fix the build on Emscripten when threading is disabled
To make a non-threaded liblzma-only build work with WASI SDK, <signal.h>
and mythread_sigmask() were omitted from mythread.h in the commit 81db3b889830. This broke non-threaded full build with Emscripten because
src/xz/signals.c needs mythread_sigmask() (liblzma-only build was fine).
If __wasm__ is defined, omit <signal.h> and mythread_sigmask() in
non-threaded builds only when __EMSCRIPTEN__ isn't defined.
Reported-by: Marcus Tillmanns Thanks-to: ChanTsune Fixes: https://github.com/tukaani-project/xz/issues/161 Fixes: 81db3b889830 ("mythread.h: Disable signal functions in builds targeting Wasm + WASI.")
projects / thirdparty / xz.git / log
