Paul Eggert [Sun, 23 Nov 2025 17:44:03 +0000 (09:44 -0800)]
Support gnulib-style timestamps in checkpoint logs
* gnulib.modules: Add nstrftime-limited, time_rz. Sort.
* src/checkpoint.c: Include <strftime.h>.
(format_checkpoint_string): Use nstrftime instead of strftime.
Also fix an obscure bug on platforms that lack tm_gmtoff+tm_zone by
calling tzalloc on those platforms; if it fails, fall back on gmtime.
Also, use fwrite instead of fprintf, since we typically know the
length already and this gives us a more-accurate byte count
in case there are partial writes.
Paul Eggert [Sat, 15 Nov 2025 23:49:23 +0000 (15:49 -0800)]
Prefer countof to sizeof / sizeof
C2y plans to introduce a new countof operator that will be
convenient for GNU tar, so start using it now via Gnulib.
* gnulib.modules: Add stdcountof-h.
* lib/wordsplit.c, src/buffer.c, src/suffix.c, src/tar.c:
Include stdcountof.h, and prefer countof (X) to sizeof X / sizeof *X.
Paul Eggert [Sat, 15 Nov 2025 21:38:44 +0000 (13:38 -0800)]
Port to compilers where COMMON_INLINE is static
Problem found with clang 15.0 on CheriBSD.
* src/names.c (namelist_match, register_match):
Now plain static, not static COMMON_INLINE, since the later
could mean the declaration is ‘static static’ which is not allowed.
Paul Eggert [Sat, 15 Nov 2025 20:55:06 +0000 (12:55 -0800)]
Skip sparse test on non-sparse file systems
* tests/sparse05.at (listing sparse files bigger than 2^33 B):
If there were problems generating BIGFILE remove it,
as it has likely exhausted the file system.
Problem found on Darwin 21.6 APFS.
Paul Eggert [Sat, 15 Nov 2025 09:05:05 +0000 (01:05 -0800)]
Adjust to Gnulib strftime changes for macOS
Stop using the fprintftime module, as as with recent Gnulib changes
it breaks the build on macOS, and fixing this would drag in threading
libraries and macOS-specific libraries that are overkill for tar.
Instead, just use strftime; that’s good enough here and arguably
better in case someone attacks tar with a huge time format string.
* gnulib.modules: Remove fprintftime.
* src/checkpoint.c: Do not include fprintftime.
(format_checkpoint_string): Always output some useful info (a decimal
seconds count), even if localtime fails. Do not output more than
256 bytes of time info, as that’s likely a DoS attack. Stick with
plain strftime, as fprintftime’s extra features are overkill here.
Paul Eggert [Sat, 15 Nov 2025 08:55:47 +0000 (00:55 -0800)]
Port intmax macro to strict C11
Needed for Apple clang version 14.0.0 (clang-1400.0.29.202).
* src/common.h (intmax): Define macro only if _Generic works, and
use _Generic in it rather that using EXPR_SIGNED. This is needed
to make the first argument of verify_expr an integer constant
expression, which is required for strict C11. Although GCC is
smart enough to treat (1 ? 0 : V) as an integer constant
expression even if V is an integer variable, C11 does not require
support for this sort of thing.
Paul Eggert [Fri, 14 Nov 2025 06:59:24 +0000 (22:59 -0800)]
Do not create empty placeholder files
* src/extract.c (HAVE_BIRTHTIME, BIRTHTIME_EQ, struct delayed_link)
(delayed_link_table, delayed_link_head delayed_link_tail)
(struct string_list, dl_hash, dl_compare, find_direct_ancestor)
(find_delayed_link_source, create_placeholder_file, apply_delayed_link)
(apply_delayed_links): Remove. All uses removed.
(struct delayed_set_stat): New member metadata_set,
replacing after_links. All uses changed.
(apply_nonancestor_delayed_set_stat): Arg METADATA_SET replaces
the old AFTER_LINKS. All callers changed.
(extract_archive): Do not worry about "..", since openat2
now does that for us.
* src/names.c (first_dot_dot): Remove. All uses removed.
Paul Eggert [Thu, 13 Nov 2025 21:44:10 +0000 (13:44 -0800)]
Use openat2 to jailify the extraction directory
This addresses CVE-2025-45582.
* gnulib.modules: Add openat2.
* src/misc.c (open_subdir): New static function.
(fdbase_opendir): Use it.
* src/tar.c (open_searchdir_how): New var, replacing and
augmenting open_searchdir_flags. All uses changed.
* tests/extrac31.at: New file.
* tests/Makefile (TESTSUITE_AT), tests/testuite.at: Add it.
Paul Eggert [Thu, 13 Nov 2025 21:18:27 +0000 (13:18 -0800)]
Use fewer flags when opening directories
This doesn’t change behavior; it is a refactoring for
compatibility with a future patch that will use Linux’s
openat2 syscall, which is pickier about flags.
* src/tar.c (decode_options): When searching directories,
do not use O_NOCTTY, O_NONBLOCK, and O_NOATIME.
openat2 rejects all three flags if O_PATH is used.
The first two flags are definitely irrelevant for directories,
and O_NOATIME probably doesn’t matter either.
Paul Eggert [Thu, 13 Nov 2025 16:15:26 +0000 (08:15 -0800)]
Improve performance of relative opendir
* src/misc.c (fdbase_opendir): When the new directory is a
subdirectory of the old one, open relative to the old one rather
than relative all the way back to chdir_fd, and if that open fails
for a non-EMFILE reason, keep rather than discard the old directory.
Paul Eggert [Thu, 13 Nov 2025 01:33:11 +0000 (17:33 -0800)]
Cache parent directories
Although this might help (or hurt) performance, the main
motivation is to make it easier in future commits
to prevent tarballs from escaping the extraction directory.
* src/common.h: (BADFD): New constant.
(struct fdbase): New type.
* src/create.c (dump_file0): Use parent->fd instead of caching
it into a local, as the latter approach is now awkward.
* src/extract.c (extract_link): Don’t save errno unless needed.
* src/misc.c (safer_rmdir): New arg F. All callers changed.
(maybe_backup_file): Construct full after_backup_name, now
that find_backup_file_name no longer does that for us.
(chdir_fd): Now static not extern, as other modules now use fdbase.
(fdbase_cache): New static var.
(fdbase_clear): New function. Call it whenever removing
or renaming directories or symlinks to directories.
(fdbase_opendir): New static function.
(fdbase, fdbase1): New functions. Call them whenever the
code formerly passed chdir_fd to a syscall.
Paul Eggert [Tue, 11 Nov 2025 16:08:58 +0000 (08:08 -0800)]
Prefer issymlinkat
* gnulib.modules: Add issymlinkat, already an indirect dependency.
* src/extract.c: Include issymlink.h.
(is_directory_link, open_output_file):
Prefer issymlinkat to doing it by hand.
Paul Eggert [Sat, 8 Nov 2025 00:26:32 +0000 (16:26 -0800)]
Refactor to avoid duplication in "./" scanning
* src/exclist.c (excluded_name):
* src/misc.c (normalize_filename_x, must_be_dot_or_slash)
(chdir_arg):
Use dotslash or dotslashlen instead of doing things by hand.
* src/misc.c (slashlen, dotslashlen): New functions.
(safer_rmdir): Do not worry about unlinkat with AT_REMOVEDIR
succeeding on ".", as POSIX prohibits it, and it does not succeed
on any known platform. This simplifies the file name test.
Continue to worry about "/" though, as POSIX does allow
it to be removed.
Paul Eggert [Tue, 28 Oct 2025 02:40:47 +0000 (19:40 -0700)]
chdir_id refactoring
This prepares for future changes that need directory IDs.
* src/common.h (struct chdir_id): New struct.
* src/extract.c (extract_dir): Use chdir_id to avoid duplicate stats.
* src/misc.c (struct wd): New member ID.
(grow_wd): New function, extracted from chdir_arg and that
also initializes id.err.
(chdir_arg): Use it. Initialize id.err.
(chdir_id): New function.
Paul Eggert [Sat, 8 Nov 2025 22:14:27 +0000 (14:14 -0800)]
Use Gnulib gendocs instead of our own
I ported our fix into Gnulib so there’s no longer a need
for a separate copy.
* doc/Makefile.am (GENDOCS): Now in ../build-aux, not here.
* doc/gendocs.sh, doc/gendocs_template: Remove.
* gnulib.modules: Add gendocs.
Paul Eggert [Tue, 19 Aug 2025 00:14:13 +0000 (17:14 -0700)]
Avoid overrun when converting ns-resolution timestamps to text
Caught by gcc -fsanitize=address.
Inspired by Matthias Andree’s bug report in:
https://lists.gnu.org/r/bug-tar/2025-08/msg00019.html
though I found this bug via a simple "make check"
with sanitization enabled.
* src/common.h (TIMESPEC_STRSIZE_BOUND):
Make room for leading '-', needed in addition to the '-' room
supplied by SYSINT_BUFSIZE due to the way code_timespec works.
Paul Eggert [Sat, 9 Aug 2025 23:10:49 +0000 (16:10 -0700)]
Use Gnulib’s same-inode module
This is more portable to non-POSIX systems.
However, don’t bother trying to port to systems
where st_ino is not a scalar of type dev_t,
as these systems no longer seem to be active targets
and it’s not worth the maintenance hassle.
* gnulib.modules: Add same-inode, now that we use it
explicitly rather than indirectly.
* src/compare.c (diff_link):
* src/create.c (compare_links, restore_parent_fd):
* src/incremen.c (compare_directory_meta, procdir):
* src/extract.c (dl_compare, repair_delayed_set_stat)
(apply_nonancestor_delayed_set_stat, extract_link)
(apply_delayed_link):
* src/names.c (add_file_id):
* src/system.c (sys_file_is_archive, sys_detect_dev_null_output):
Include same-inode.h, and prefer its macros and functions
to doing things by hand.
* src/create.c (struct link):
* src/extract.c (struct delayed_set_stat, struct delayed_link):
* src/incremen.c (struct directory):
* src/names.c (struct file_id_list):
Rename members to st_dev and st_ino so that SAME_INODE and
PSAME_INODE can be used on the type. All uses changed.
* src/system.c (sys_compare_links): Remove.
All uses replaced by psame_inode.
Collin Funk [Fri, 8 Aug 2025 03:42:14 +0000 (20:42 -0700)]
manual: remove '.info' suffix in manual names passed to @xref
Texinfo 7.2 began warning about the '.info' suffix in the manual names
passed to @xref and similar commands. They eventually plan to stop
stripping the '.info' suffix internally which will lead to broken links
in the manuals without this change.
* doc/tar.texi (files): Remove '.info' suffix from manual name.
Paul Eggert [Sun, 27 Jul 2025 04:41:23 +0000 (21:41 -0700)]
--no-overwrite-dir no overwrite even temporarily
Problem and fix reported by Pavel Cahyna in
https://lists.gnu.org/r/bug-tar/2025-01/msg00000.html
* src/extract.c (extract_dir): With --no-overwrite-dir,
skip the chmod if the directory already exists.
* tests/extrac23.at (--no-overwrite-dir on empty directory):
Move the part of the test that looks at a nonempty directory ...
* tests/extrac30.at: ... to this new file, because the test now
must be run as non-root. Adjust the test to match the new behavior.
* tests/Makefile.am (TESTSUITE_AT), tests/testsuite.at: Add it.
Paul Eggert [Sat, 26 Jul 2025 07:47:48 +0000 (00:47 -0700)]
Use flexible array member in struct dumpdir
* src/incremen.c: Include flexmember.h.
(struct dumpdir): contents is now a flexible member, not a pointer.
This is more idiomatic and slightly more efficient.
(dumpdir_create0): Adjust to the new struct dumpdir layout.
Paul Eggert [Sat, 26 Jul 2025 07:27:01 +0000 (00:27 -0700)]
Avoid undefined behavior in magic checking
* src/buffer.c (check_compressed_archive):
* src/list.c (read_header, decode_header):
Use memcmp, not strcmp, when looking for magic strings in
headers, since input headers are not guaranteed to be
strings and strcmp has undefined behavior otherwise.
Paul Eggert [Sat, 26 Jul 2025 03:39:32 +0000 (20:39 -0700)]
Port more code to UBSan, and fix alignment bug
Problem with extract_file reported by Kirill Furman in:
https://lists.gnu.org/r/bug-tar/2025-07/msg00003.html
Since the UBSan thing seems to be a recurring issue,
I fixed other instances of the problem that I found.
Also, I noticed that the same line of code had another failure to
conform to C23’s rules for pointers (an alignment issue not caught
by UBSan), so I fixed that too. None of these issues matter on
practical production hosts.
* src/common.h (charptr): New function.
* src/buffer.c (available_space_after, short_read, flush_archive)
(backspace_output, try_new_volume, simple_flush_read)
(_gnu_flush_read, _gnu_flush_write):
* src/compare.c (read_and_process):
* src/create.c (write_eot, write_gnu_long_link)
(dump_regular_file, dump_dir0):
* src/extract.c (extract_file):
* src/incremen.c (get_gnu_dumpdir):
* src/list.c (read_header):
* src/sparse.c (sparse_dump_region, sparse_extract_region):
* src/system.c (sys_write_archive_buffer)
(sys_child_open_for_compress, sys_child_open_for_uncompress):
* src/update.c (append_file, update_archive):
Use it.
* src/buffer.c (set_next_block_after): Arg is now void *,
not union block *, since it need not be a valid union block * pointer
and this can matter on unusual or debugging implementations.
Turn a loop into an if so that the code is O(1) not O(N).
Paul Eggert [Thu, 12 Jun 2025 07:20:52 +0000 (00:20 -0700)]
Port short_read to UBSan
Problem reported by Kirill Furman in:
https://lists.gnu.org/r/bug-tar/2025-06/msg00002.html
* src/buffer.c (short_read): Use (char *) record_start,
instead of record_start->buffer, to avoid undefined behavior
accessing past end of buffer. In practice the undefined
behavior is harmless unless running with -fsanitize=undefined
or a similarly-picky implementation.
* src/names.c (namelist_match_from): New function.
(namelist_match): Rewrite as a wrapper over it.
(register_match): New function.
(name_match)" Update all possible matches in the name list.
* tests/extrac29.at: New test.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
Handle directory members consistently when listing and when extracting.
* src/list.c (skim_member): Recognize directory members using
the same rules as during extraction.
* tests/skipdir.at: New testcase.
* tests/testsuite.at: Add new test.
* tests/Makefile.am: Likewise.
Anssi Hannula [Wed, 6 Nov 2024 10:43:32 +0000 (12:43 +0200)]
Fix missing type in mknodat() mode argument
Per POSIX, the type of the file to be created should be OR'ed to the
`mode` argument of mknodat().
However, set_xattr() creates an empty file using mknodat() and does not
do that.
E.g. Linux kernel considers zero type as S_IFREG, so the code works on
most systems.
However, e.g. fakeroot, at least up to the current v1.36, does not
consider 0 as S_IFREG, instead creating an invalid file, causing tar to
enter an infinite retry loop when trying to create a file with xattrs
under fakeroot.
Since set_xattr is used only when extracting regular files, fix that
by ORing its mode argument with S_IFREG.
Paul Eggert [Tue, 29 Apr 2025 21:05:59 +0000 (14:05 -0700)]
Port to recent Gnulib hash_remove
Problem reported by Bruno Haible in:
https://lists.gnu.org/r/bug-tar/2025-04/msg00003.html
* src/incremen.c (remove_directory): hash_delete → hash_remove.
* src/extract.c (update_interdir_set_stat): New function.
(extract_dir): If the directory already exists, check if it
has been created as intermediate directory earlier. If so,
update its delayed_set_stat data from archive.
* tests/Makefile.am: Add new testcase.
* tests/testsuite.at: Add new testcase.
* tests/extrac28.at: New file.
Paul Eggert [Wed, 6 Nov 2024 18:18:13 +0000 (10:18 -0800)]
Fix bad pointer usage in xsparse.c
* scripts/xsparse.c (read_xheader): Avoid undefined behavior by
accessing via null pointer sparse_map or out of its bounds when
the input is invalid. This means we no longer need the ‘expect’
local, so omit it for simplicity.
Paul Eggert [Wed, 6 Nov 2024 18:05:23 +0000 (10:05 -0800)]
Port xsparse.c to AIX
* scripts/xsparse.c (emalloc): Do not report failure when malloc
(0) returns NULL, as it does on AIX. Simply return a null
pointer; that’s good enough for xsparse.c.
Paul Eggert [Wed, 6 Nov 2024 18:02:02 +0000 (10:02 -0800)]
Fix xsparse.c big heap allocation bugs
* scripts/xsparse.c (expand_sparse): Read into auto buffer, not heap.
The heap code was wrong for two reasons: it called malloc just once
in the try-again loop, and even when it succeeded it could have
left so few bytes available in the heap that later stdio calls
could fail. Reading into the auto buffer might be a bit slower
but speed is not an issue here and it’s better to be simple.
Paul Eggert [Sat, 2 Nov 2024 20:42:02 +0000 (13:42 -0700)]
Add LG_BLOCKSIZE to omit some *, % ops
* src/buffer.c (_flush_write, short_read, seek_archive)
(_gnu_flush_write):
* src/create.c (write_gnu_long_link, dump_regular_file)
(dump_dir0):
* src/delete.c (write_recent_bytes, flush_file)
(delete_archive_members):
* src/list.c (read_header):
* src/sparse.c (sparse_dump_region, sparse_extract_region)
(pax_dump_header_1):
* src/tar.c (parse_opt):
* src/update.c (append_file):
Prefer shifting and masking to dividing and remaindering by
BLOCKSIZE. This reclaims some compiler optimizations lost
by our recent preference for signed integers.
* src/tar.h (LG_BLOCKSIZE): New constant, for shifting.
Paul Eggert [Sat, 2 Nov 2024 20:06:47 +0000 (13:06 -0700)]
Improve sparse I/O performance
* src/sparse.c (sparse_dump_region, sparse_extract_region):
Don’t insist on reading and writing sparse files 512
bytes at a time. This resulted in a 4× to 6× performance
improvement on my platform.
Paul Eggert [Sat, 2 Nov 2024 18:52:28 +0000 (11:52 -0700)]
Simplify read_incr_db_01 malloc
* src/incremen.c (read_incr_db_01): Replace arg initbuf with two
args pbuf and pbufsize so that we can simplify memory allocation.
Caller changed. Omit now-unnecessary free, xstrdup, strlen.
Paul Eggert [Sat, 2 Nov 2024 16:54:10 +0000 (09:54 -0700)]
Count short read slop when seeking
* src/buffer.c (short_read_slop): New static var.
(get_archive_status): Treat anything other than fifos and sockets
as potentially seekable; they’ll tell us if they aren’t, whereas
fifos and sockets cannot be seekable. Check named files for
initial offset too, to deal with names like /dev/stdin.
Do not worry about start_offset’s value if !seekable_archive,
as it won’t be used. Use short_read_slop.
(short_read, try_new_volume, simple_flush_read, _gnu_flush_read):
Set short_read_slop.
Paul Eggert [Sat, 2 Nov 2024 02:49:02 +0000 (19:49 -0700)]
Prefer other types to int in names.c
* src/names.c (uname_to_uid, gname_to_gid, handle_option)
(make_file_name): Prefer bool for boolean.
(struct name_elt, read_name_from_file): Prefer char for char.
(handle_option): Invert sense of return value, for clarity.
All uses changed.
(merge_sort_sll, merge_sort, collect_and_sort_names):
Don’t assume list length fits in int. Use intptr_t not idx_t,
since the bound is the size of all memory rather than one array.
Paul Eggert [Sat, 2 Nov 2024 02:09:44 +0000 (19:09 -0700)]
Prefer other types to int in misc.c
* src/misc.c (quote_copy_string, tar_savedir):
Use bool for booleans. All uses changed.
(quote_copy_string): Use char for chars.
(unquote_string): Return void, since nobody uses return value.
(unquote_string): Check for overflow in escapes like \777.
(wdcache): Now array of idx_t not int, since in theory it
might contain values greater than INT_MAX. All uses changed.
Paul Eggert [Sat, 2 Nov 2024 01:45:00 +0000 (18:45 -0700)]
Fix some uses of int in list.c
* src/list.c (decode_xform): Last arg is now int, not a void *
pointer to that int. All uses changed.
(enforce_one_top_level): Don’t assume string length fits in int.
(transform_stat_info): Prefer char to int for typeflag.
All uses changed.
(decode_header): Prefer bool for booleans. All uses changed.
(ugswidth): Now idx_t, not int, since in theory it could
exceed INT_MAX. All uses changed.
(simple_print_header, print_for_mkdir): Don’t assume printf length
fits in int, and similarly for length of user or group name.
* src/transform.c (transform_name_fp): Last arg is now int, not void *.
All uses changed.
Paul Eggert [Fri, 1 Nov 2024 21:27:21 +0000 (14:27 -0700)]
Prefer other types to int in incremen.c
* src/incremen.c (struct dumpdir_iter, dumpdir_first)
(read_incr_db_01, dumpdir_ok, list_dumpdir):
Prefer bool to int for booleans. All uses changed.
(read_incr_db_01): Don’t assume getline returns <= INT_MAX.
(dumpdir_ok): Prefer char to int for chars.
Paul Eggert [Fri, 1 Nov 2024 21:15:09 +0000 (14:15 -0700)]
Prefer other types to int in extract.c
* src/extract.c (fd_chmod, extract_chdir, open_output_file)
(extract_file, extract_link, extract_symlink, extract_node)
(extract_fifo, tar_extractor_t, pepare_to_extract): Prefer char to
int for typeflag, since it’s a char. All uses changed.
(fd_chmod): Use clearer code for errno.
(extract_dir, extract_file, create_placeholder_file, extract_link)
(extract_symlink, extract_node, extract_fifo, tar_extractor_t):
Return bool true for success, false for failure. All uses changed.
(open_output_file): Prefer bool for boolean.
(prepare_to_extract): Simplify by returning the extractor a null
pointer, rather than storing through a pointer to an extractor.
Paul Eggert [Fri, 1 Nov 2024 18:04:39 +0000 (11:04 -0700)]
Check for checkpoint string overflow
It’s very unlikely, but would lead to undefined behavior.
* src/checkpoint.c (format_checkpoint_string): Accept and return
intmax_t, not idx_t. All callers changed. Check for integer
overflow by using add_printf. If overflow occurs, don’t bother
with extending width.
Paul Eggert [Fri, 1 Nov 2024 17:37:39 +0000 (10:37 -0700)]
Prefer int to idx_t for some small sizes
* src/create.c (max_octal_val, to_octal, tar_copy_str)
(tar_name_copy_str, to_base256, to_chars_subst, to_chars)
(gid_to_chars, major_to_chars, minor_to_chars, mode_to_chars)
(off_to_chars, time_to_chars, uid_to_chars, string_to_chars)
(split_long_name, write_ustar_long_name, simple_finish_header):
* src/list.c (from_header, gid_from_header, major_from_header)
(minor_from_header, mode_from_header, off_from_header)
(time_from_header, uid_from_header):
Prefer int to idx_t where either will do because the buffer sizes
are known to be small, as this can be a performance win on 32-bit
platforms. Also, in a few cases the values were negative, whereas
idx_t is supposed to be nonnegative.