If a pax attribute has a 0 length value and no newline, the tar reader
gets out of sync with block alignment.
This happens because the pax parser assumes that variable value_length
(which includes the terminating newline) is at least 1. To get the
real value length, 1 is subtracted. This result is subtracted from
extsize, which in this case would lead to `extsize -= -1`, i.e.
the remaining byte count is increased.
Such an unexpected calculation leads to an off-by-one when skipping
to the next block. In supplied test case, bsdtar complains that the
checksum of the next block is wrong. Since the tar parser was not
properly 512 bytes aligned, this is no surprise.
Gracefully handle such a case like GNU tar does and warn the user that
an invalid attribute has been encountered.
Zhaofeng Li [Sat, 24 May 2025 19:45:18 +0000 (13:45 -0600)]
tar: Reset accumulated header state after reading macOS metadata blob
AppleDouble extension entries are present as separate files immediately
preceding the corresponding real files. In libarchive, we process the
entire metadata file (headers + data) as if it were a header in the real
file. However, the code forgets to reset the accumulated header state
before parsing the real file's headers. In one code path, this causes
the metadata file's name to be used as the real file's name.
Specifically, this can be triggered with a tar containing two files:
1. A file named `._badname` with pax header containing the `path` attribute
2. A file named `goodname` _with_ a pax header but _without_ the `path` attribute
libarchive will list one file, `._badname` containing the data of `goodname`.
This code is pretty brittle and we really should let the client deal with
it :(
The count fields are merely used to check if a list is empty or not.
A check for first being not NULL is sufficient and is already in
place while iterating over the linked elements (count is not used).
The operations for key and node comparison depend on the platform
libarchive is compiled for. Since these values do not change
during runtime, set them only once during initialisation.
Further simplify the code by declaring only one "rb_ops" with
required functions based on platform.
The cygwin FAQ states that __CYGWIN__ is defined when building for a
Cygwin environment. Only a few test files check (inconsistently) for
CYGWIN, so adjust them to the recommended __CYGWIN__ definition.
Cast address of "version" to BYTE pointer for CryptGetProvParam.
Fix "major" variable assignment for picky compilers like MSVC.
The "length" variable is an in/out variable. It must be set to the size
of available memory within "version". Right now it is undefined behavior
and 0 would crash during runtime.
dependabot[bot] [Tue, 20 May 2025 08:19:56 +0000 (10:19 +0200)]
CI: Bump the all-actions group across 1 directory with 4 updates (#2623)
Bumps the all-actions group with 4 updates:
`actions/checkout` from 4.2.1 to 4.2.2
`actions/upload-artifact` from 4.4.3 to 4.6.2
`github/codeql-action` from 3.26.12 to 3.28.18
`ossf/scorecard-action` from 2.4.0 to 2.4.1
Rose [Sat, 17 May 2025 23:35:22 +0000 (19:35 -0400)]
Fatal if field[0].start or field[0].end is null
We should not get here, but given that the check exists, we should not let it happen if this is NULL because otherwise we just dereference it later on.
Nicholas Vinson [Sun, 13 Apr 2025 11:33:43 +0000 (07:33 -0400)]
Copy ae digests to mtree_entry
Copy ae digests to mtree_entry. This simplifies porting non-archive
formats to archive formats while preserving supported message
digests specifically in cases where recomputing digests is not
viable.
Signed-off-by: Nicholas Vinson <nvinson234@gmail.com>
The size_t to int conversion is especially required on Windows systems
to support their int-based functions. These variables should be properly
checked before casts. This avoids integer truncations with large
strings.
I prefer size_t over int for sizes and adjusted variables to size_t
where possible to avoid casts.
If vsnprintf fails with errno EOVERFLOW, the results are very platform
dependent but never useful. The implementation in glibc fills bytes with
blanks, FreeBSD fills them with zeros, OpenBSD and Windows set first
byte to '\0'.
Just stop processing and don't print anything, which makes it follow
the OpenBSD and Windows approach.
The stack buffer is never cleared, which can become an issue depending
on vsnprintf implementation's behavior if -1 is returned. The code
would eventually fall back to stack buffer which might be not
nul terminated.
Zhaofeng Li [Thu, 15 May 2025 12:08:14 +0000 (06:08 -0600)]
bsdtar: Support `--mtime` and `--clamp-mtime` (#2601)
Hi,
This PR adds support for setting a forced mtime on all written files
(`--mtime` and `--clamp-mtime`) in bsdtar.
The end goal will be to support all functionalities in
<https://reproducible-builds.org/docs/archives/#full-example>, namely
`--sort` and disabling other attributes (atime, ctime, etc.).
Fixes #971.
## History
- [v1](https://github.com/zhaofengli/libarchive/tree/forced-mtime-v1):
Added `archive_read_disk_set_forced_mtime` in libarchive. As a result,
it was only applied when reading from the filesystem and not from other
archives.
- [v2](https://github.com/zhaofengli/libarchive/tree/forced-mtime-v2):
Refactored to apply the forced mtime in `archive_write`.
- v3 (current): Reduced libarchive change to exposing
`archive_parse_date`, moved clamping logic into bsdtar.
---------
Signed-off-by: Zhaofeng Li <hello@zhaofeng.li> Co-authored-by: Dustin L. Howett <dustin@howett.net>
A filter block size must not be larger than the lzss window, which is
defined
by dictionary size, which in turn can be derived from unpacked file
size.
While at it, improve error messages and fix lzss window wrap around
logic.
rar: Fix double free with over 4 billion nodes (#2598)
If a system is capable of handling 4 billion nodes in memory, a double
free could occur because of an unsigned integer overflow leading to a
realloc call with size argument of 0. Eventually, the client will
release that memory again, triggering a double free.
mehrabiworkmail [Fri, 9 May 2025 17:21:32 +0000 (10:21 -0700)]
7z sfx overaly detection (#2088)
To detect 7z SFX files, libarchive currently searches for the 7z header
in a hard-coded addr range of the PE/ELF file
(specified via macros SFX_MIN_ADDR and SFX_MAX_ADDR). This causes it to
miss SFX files that may stray outside these values (libarchive fails to
extract 7z SFX ELF files created by recent versions of 7z tool because
of this issue). This patch fixes the issue by finding a more robust
starting point for the 7z header search: overlay in PE or the .data
section in ELF. This patch also adds 3 new test cases for 7z SFX to
libarchive.
7zip reader: add test for POWERPC filter support for LZMA compressor (#2460)
This new test archive contains a C hello world executable built like so
on a ubuntu 24.04 machine:
```
int main(int argc, char *argv[]) {
printf("hello, world\n");
return 0;
}
```
`powerpc-linux-gnu-gcc hw.c -o hw-powerpc -Wall`
The test archive that contains this executable was created like so,
using 7-Zip 24.08: `7zz a -t7z -m0=lzma2 -mf=ppc
libarchive/test/test_read_format_7zip_lzma2_powerpc.7z hw-powerpc`
The new test archive is required because the powerpc filter for lzma is
implemented in liblzma rather than in libarchive.
xar: add xmllite support to the XAR reader and writer (#2388)
This commit adds support for reading and writing XAR archives on Windows
using the built-in xmllite library. xmllite is present in all versions
of Windows starting with Windows XP.
With this change, no external XML library (libxml2, expat) is required
to read or produce XAR archives on Windows.
xmllite is a little bit annoying in that it's entirely a COM API--the
likes of which are annoying to use from C.
Tim Kientzle [Fri, 9 May 2025 11:36:05 +0000 (04:36 -0700)]
Polish for GNU tar format reading/writing (#2455)
A few small tweaks to improve reading/writing of the legacy GNU tar
format.
* Be more tolerant of redundant 'K' and 'L' headers
* Fill in missing error messages for redundant headers
* New test for reading archive with redundant 'L' headers
* Earlier identification of GNU tar format in some cases
These changes were inspired by Issue #2434. Although that was determined
to not technically be a bug in libarchive, it's relatively easy for
libarchive to tolerate duplicate 'K' and 'L' headers and we should be
issuing appropriate error messages in any case.
The refactoring of https://github.com/libarchive/libarchive/pull/2553
introduced three issues:
1. Introduction of a modifiable global static variable
This violates the goal of having no global variables as stated in [the
README.md](https://github.com/libarchive/libarchive/blob/b6f6557abb8235f604eced6facb42da8c7ab2a41/README.md?plain=1#L195)
which in turn leads to concurrency issues. Without any form of mutex
protection, multiple threads are not guaranteed to see the correct
min/max values. Since these are not needed in regular use cases but only
in edge cases, handle them in functions with local variables only.
Also the global variables are locale-dependent which can change during
runtime. In that case, future calls leads to issues.
2. Broken 32 bit support
The writers for zip and others affected by the previously mentioned PR
and test-suite on Debian 12 i686 are broken, because the calculation of
maximum MS-DOS time is not possible with a 32 bit time_t. Treat these
cases properly.
3. Edge case protection
Huge or tiny int64_t values can easily lead to unsigned integer
overflows. While these do not affect stability of libarchive, the
results are still wrong, i.e. are not capped at min/max as expected.
In total, the functions are much closer to their original versions again
(+ more range checks).
Make sure that size_t casts do not truncate the value of packed_size on
32 bit systems since it's 64 bit. Extensions to RAR format allow 64 bit
values to be specified in archives.
Also verify that 64 bit signed arithmetics do not overflow.
It is possible to trigger a call stack overflow by repeatedly entering
the rar_read_ahead function. In normal circumstances, this recursion is
optimized away by common compilers, but default settings with MSVC keep
the recursion in place. Explicitly turn the recursion into a goto-loop
to avoid the overflow even with no compiler optimizations.
Reset avail_in and next_in if the next entry of a split archive is
parsed to always update its internal structure to access next bytes when
cache runs empty.
rar: Support large headers on 32 bit systems (#2596)
Support header sizes larger than 32 bit even on 32 bit systems, since
these normally have large file support. Otherwise an unsigned integer
overflow could occur, leading to erroneous parsing on these systems.
Unify spacing in archive_read_support_format_rar.c (#2590)
Most source files use tabs instead of spaces, but
archive_read_support_format_rar.c uses spaces most of the time. A few
lines contain a mixture of tabs and spaces, which leads to poorly
formatted output with many default settings.
Unify the style. No functional change and preparation for upcoming
changes to get rid of white space diffs.
Tim Kientzle [Sat, 26 Apr 2025 23:10:45 +0000 (16:10 -0700)]
Improve support for AFIO 64-bit inode values (#2589)
PR #2258 hardened AFIO parsing by converting all inode values >= 2^63 to
zero values. This turns out to be problematic for filesystems that use
very large inode values; it results in all such files being viewed as
hardlinks to each other.
PR #2587 partially addressed this by instead considering inode values >=
2^63 as invalid and just ignoring them. This prevented the accidental
hardlinking, but at the cost of losing all hardlinks that involved large
inode values.
This PR further improves things by stripping the high order bit from
64-bit inode values in the AFIO parser. This allows them to be mostly
preserved and should allow hardlinks to get properly processed in the
vast majority of cases. The only false hardlinks would be when there are
inode values that differ exactly in the high order bit, which should be
very rare.
A full solution will require expanding inode handling to use unsigned
64-bit values; we can't do that without a major version bump, but this
PR also sets the stage for migrating inode support in a future
libarchive 4.0.
Brian Campbell [Sat, 26 Apr 2025 04:11:19 +0000 (05:11 +0100)]
Fix overflow in build_ustar_entry (#2588)
The calculations for the suffix and prefix can increment the endpoint
for a trailing slash. Hence the limits used should be one lower than the
maximum number of bytes.
Without this patch, when this happens for both the prefix and the
suffix, we end up with 156 + 100 bytes, and the write of the null at the
end will overflow the 256 byte buffer. This can be reproduced by running
```
mkdir -p foo/bar
bsdtar cvf test.tar foo////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////bar
```
when bsdtar is compiled with Address Sanitiser, although I originally
noticed this by accident with a genuine filename on a CHERI capability
system, which faults immediately on the buffer overflow.
But signed type is used in libarchive and there were some fuzzing issues
with it, https://github.com/libarchive/libarchive/pull/2258 converts
negative `ino` to `0`, which is actually a reserved inode value, but
more importantly it was still setting `AE_SET_INO` flag and later on
hardlink detection will treat all `0` on same `dev` as hardlinks to each
other if they have some hardlinks.
This showed up in BuildBarn FUSE filesystem
https://github.com/buildbarn/bb-remote-execution/issues/162 which has
both
- setting number of links to a big value
- generating random inode values in full uint64 range
Which in turn triggers false-positive hardlink detection in `bsdtar`
with high probability.
Let's mitigate it
- don't set `AE_SET_INO` on negative values (assuming rest of code isn't
ready to correctly handle full uint64 range)
- check that `ino + dev` are set in link resolver
Make sure to not skip past end of file for better error messages. One
such example is now visible with rar testsuite. You can see the
difference already by an actually not useless use of cat:
```
$ cat .../test_read_format_rar_ppmd_use_after_free.rar | bsdtar -t
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Truncated input file (needed 119 bytes, only 0 available)
bsdtar: Error exit delayed from previous errors.
```
compared to
```
$ bsdtar -tf .../test_read_format_rar_ppmd_use_after_free.rar
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Archive entry has empty or unreadable filename ... skipping.
bsdtar: Error exit delayed from previous errors.
```
Since the former cannot lseek, the error is a different one
(ARCHIVE_FATAL vs ARCHIVE_EOF). The piped version states explicitly that
truncation occurred, while the latter states EOF because the skip past
the end of file was successful.
tar: Improve LFS support on 32 bit systems (#2582)
The size_t data type is only 32 bit on 32 bit sytems while off_t is
generally 64 bit to support files larger than 2 GB.
If an entry is declared to be larger than 4 GB and the entry shall be
skipped, then 32 bit systems truncate the requested amount of bytes.
This leads to different interpretation of data in tar files compared to
64 bit systems.
Solves a Windows compile issue when OpenSSH/mbedTLS/Nettle is activated
and on the build system's paths by making the Windows API backend higher
priority on Windows (meaning that only RIPEMD160 will use
OpenSSH/mbedTLS/Nettle anymore).
If a warc archive claims to have more than INT64_MAX - 4 content bytes,
the inevitable failure to skip all these bytes could lead to parsing
data which should be ignored instead.
The test case contains a conversation entry with that many bytes and if
the entry is not properly skipped, the warc implementation would read
the conversation data as a new file entry.
The skip functions are limited to 1 GB for cases in which libarchive
runs on a system with an off_t or long with 32 bits. This has negative
impact on 64 bit systems.
Instead, make sure that _all_ subsequent functions truncate properly.
Some of them already did and some had regressions for over 10 years.
Tests pass on Debian 12 i686 configured with --disable-largefile, i.e.
running with an off_t with 32 bits.
Casts added where needed to still pass MSVC builds.
Tim Kientzle [Sun, 30 Mar 2025 16:26:25 +0000 (09:26 -0700)]
Issue 2548: Reading GNU sparse entries (#2558)
My attempt to fix #2404 just made the confusion between the size of the
extracted file and the size of the contents in the tar archive worse
than it was before.
@ferivoz in #2557 showed that the confusion stemmed from a point where
we were setting the size in the entry (which is by definition the size
of the file on disk) when we read the `GNU.sparse.size` and
`GNU.sparse.realsize` attributes (which might represent the size on disk
or in the archive) and then using that to determine whether to read the
value in ustar header (which represents the size of the data in the
archive).
The confusion stems from three issues:
* The GNU.sparse.* fields mean different things depending on the version
of GNU tar used.
* The regular Pax `size` field overrides the value in the ustar header,
but the GNU sparse size fields don't always do so.
* The previous libarchive code tried to reconcile different size
information as we went along, which is problematic because the order in
which this information appears can vary.
This PR makes one big structural change: We now have separate storage
for every different size field we might encounter. We now just store
these values and record which one we saw. Then at the end, when we have
all the information available at once, we can use this data to determine
the size on disk and the size in the archive.
A few key facts about GNU sparse formats:
* GNU legacy sparse format: Stored all the relevant info in an extension
of the ustar header.
* GNU pax 0.0 format: Used `GNU.sparse.size` to store the size on disk
* GNU pax 0.1 format: Used `GNU.sparse.size` to store the size on disk
* GNU pax 1.0 format: Used `GNU.sparse.realsize` to store the size on
disk; repurposed `GNU.sparse.size` to store the size in the archive, but
omitted this in favor of the ustar size field when that could be used.
And of course, some key precedence information:
* Pax `size` field always overrides the ustar header size field.
* GNU sparse size fields override it ONLY when they represent the size
of the data in the archive.
projects / thirdparty / libarchive.git / log
