Pavel Raiskup [Tue, 31 Jul 2018 09:06:09 +0000 (12:06 +0300)]
Avoid some resource leaks
* src/incremen.c (store_rename): Free temp_name, leaked before for
each renamed directory with --listed-incremental.
* src/transform.c (add_literal_segment): Tighten arguments by
const.
(parse_transform_expr): Free 'str', leaked storage for each
--transform option before.
* src/utf8.c (utf8_convert): Deallocate buffer for failed iconv()
call so callers don't have to.
Jim Meyering [Sat, 7 Apr 2018 15:41:46 +0000 (08:41 -0700)]
--one-top-level: avoid a heap-buffer-overflow
* NEWS: Mention this.
* src/suffix.c (strip_compression_suffix): Fix string comparison guard.
Without this change, some ASAN-enabled test runs would fail with the
following. Also, strip an additional .tar suffix only if the just-
stripped suffix did not match /^\.t/".
==30815==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000002ed at pc 0x00000049d1f4 bp 0x7ffeb5906d50 sp 0x7ffeb5906500
READ of size 1 at 0x6020000002ed thread T0
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
#0 0x49d1f3 in __interceptor_strncmp /j/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:407
#1 0x5670f3 in strip_compression_suffix /j/tar/src/suffix.c:107
#2 0x575788 in decode_options /j/tar/src/tar.c:2545
#3 0x5760c0 in main /j/tar/src/tar.c:2708
#4 0x7f105090df29 in __libc_start_main ../csu/libc-start.c:308
#5 0x408629 in _start (/j/tar/src/tar+0x408629)
0x6020000002ed is located 3 bytes to the left of 6-byte region [0x6020000002f0,0x6020000002f6)
allocated by thread T0 here:
#0 0x4d0710 in __interceptor_malloc /j/gcc/libsanitizer/asan/asan_malloc_linux.cc:86
#1 0x4908ad in __interceptor_strndup /j/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:326
#2 0x5cbcbd in xstrndup /j/tar/gnu/xstrndup.c:32
#3 0x5a325b in base_name /j/tar/gnu/basename.c:57
#4 0x575772 in decode_options /j/tar/src/tar.c:2544
#5 0x5760c0 in main /j/tar/src/tar.c:2708
#6 0x7f105090df29 in __libc_start_main ../csu/libc-start.c:308
Jim Meyering [Sun, 25 Mar 2018 20:29:13 +0000 (13:29 -0700)]
tests: avoid test failure with non-ancient autoconf
Running "make check" would fail with this:
T-nonl.at:30: error: m4_divert_push: cannot change diversion
to `GROW' inside m4_expand
* tests/T-nonl.at: Use printf in place of AS_ECHO_N.
These days, printf should work for everyone.
Jim Meyering [Mon, 19 Mar 2018 04:32:19 +0000 (21:32 -0700)]
maint: avoid -Wstringop-truncation warnings from upcoming GCC8
* src/create.c (start_private_header, start_header): Convert
trivial uses of strncpy to memcpy, to avoid warnings like this:
In function 'strncpy',
inlined from 'start_private_header' at create.c:522:3:
/usr/include/bits/string_fortified.h:106:10: warning: \
'__builtin_strncpy' output truncated before terminating nul \
copying 2 bytes from a string of the same length \
[-Wstringop-truncation]
* src/buffer.c (gnu_add_multi_volume_header): Convert a use of
strncpy to memcpy, to avoid this warning:
In function 'strncpy',
inlined from 'gnu_add_multi_volume_header' at buffer.c:1782:3,
...
/usr/include/bits/string_fortified.h:106:10: error: '__builtin_strncpy'\
specified bound 100 equals destination size \
[-Werror=stringop-truncation]
Jim Meyering [Mon, 19 Mar 2018 00:59:25 +0000 (17:59 -0700)]
maint: avoid warnings from upcoming GCC8
* src/transform.c (_single_transform_name_to_obstack): Mark with
FALLTHROUGH statement rather than /* FALL THROUGH */ comment.
Only the former works with gcc-8.
* src/extract.c (maybe_recoverable): Call abort to tell gcc-8 that
this code is unreachable.
Jim Meyering [Mon, 26 Mar 2018 04:17:18 +0000 (21:17 -0700)]
tests: add coverage for new --zstd and all other compression tools
* tests/compress.at.in: New file: template from which each
per-compression-tool test is derived.
* tests/Makefile.am (TESTSUITE_AT): Add it.
(EXTRA_DIST): Add compress.at.in.
(compress-*.at): New rules to generate a file/tests for each compression tool.
* tests/testsuite.at (Compression): Add each of these generated
files in a new section.
* tests/.gitignore: Ignore the new generated files.
Pavel Raiskup [Thu, 4 Jan 2018 17:55:12 +0000 (18:55 +0100)]
tests: fix race in dirrem01 and dirrem02
Previously the '--checkpoint-action=echo' was triggered after
'--checkpoint-action=sleep=1' - so the order of events *usually*
was (for --format='gnu'):
...
1. checkpoint handler before write of 'dir/sub' member
2. one-second delay
3. stderr write: 'tar: Write checkpoint 3'
4. write the member 'dir/sub' into the archive
5. check that the member's ctime has not been changed
6. genfile's detecting 'Write checkpoint', doing unlink
...
But sometimes, the genfile was fast enough to win the race and
unlinked the directory before the member was written into the
archive (IOW, the order was 1-2-3-6-4-5). This led to the
occasional warning 'tar: dir/sub: file changed as we read it'.
Swap the order of 'sleep=1' and 'echo' actions so the genfile
utility has (hopefully) enough time to do the unlink before
writing the file into the archive (enforce 1-2-3-6-4-5 order).
* tests/dirrem01.at: Swap 'sleep=1' and 'echo' actions.
* tests/dirrem02.at: Likewise.
Paul Eggert [Sat, 18 Nov 2017 16:39:33 +0000 (08:39 -0800)]
Port to gcc -Wimplicit-fallthrough=5
* src/common.h (FALLTHROUGH): New macro, for use with gcc
-Wimplicit-fallthrough=5, which is now the default when used with
Gnulib after commit 2017-05-16T16:23:52!eggert@cs.ucla.edu
and with --enable-gcc-warnings
* src/extract.c (find_direct_ancestor): New function.
(create_placeholder_file): Set after_links member on delayed_set_stat
entries starting from the direct ancestor of the placeholder file.
* src/names.c (read_next_name): Don't unquote name read from the
file, if --verbatim-files-from option is in effect.
(names_options): improve description of --verbatim-files-from
* tests/T-null2.at: Test the change.
Fix the effect of --transform over hard link targets without the --absolute-names
* src/create.c (file_count_links): Apply safer_name_suffix to the
hard link name prior to transforming it.
* tests/xform03.at: New test case.
* tests/Makefile.am: Add xform03.at
* tests/testsuite.at: Likewise.
Pavel Raiskup [Thu, 30 Mar 2017 11:30:15 +0000 (13:30 +0200)]
Fix non-deterministic archive type detection
Due to analysis of partly uninitialized read-ahead buffer
(short_read call), we sometimes mistakenly classified very small
compressed archives as non-compressed; which in turn caused
extraction failure.
* src/buffer.c (check_compressed_archive): Don't assume that
archives smaller than BLOCKSIZE could be non-compressed, as tar
header always has at least one block.
When creating multivolume archives, the bufmap code in buffer.c
implicitly assumed that the members are stored in the archive
contiguously, ignoring the member (and eventual extended) headers
between them. This worked until the member header happened to be
at the very beginning of the volume, in which case its length was
included in the calculation of the stored size and size left to
store. Due to this, the GNUFileParts extended header contained
invalid GNU.volume.offset value, and the resulting archive failed
to extract properly.
This patch also eliminates improper listing of file part headers
as regular files, when creating multivolume posix archives with -v.
* src/buffer.c (bufmap): New member nblocks. Counts number of blocks
of file data written since reset.
(bufmap_reset): Reset nblocks to 0.
(_flush_write): Update nblocks. When computing offset difference for
bufmap_reset, count only data blocks, not headers.
(close_archive): Flush archive until all blocks are written.
(add_chunk_header): Use simple_finish_header instead of finish_header
to avoid listing chunk header as regular file in verbose mode.
* tests/multiv10.at: New test case.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Add new test.
Paul Eggert [Fri, 7 Apr 2017 01:16:51 +0000 (18:16 -0700)]
--numeric-owner now affects private headers too
Problem reported by Daniel Peebles in:
http://lists.gnu.org/archive/html/bug-tar/2017-04/msg00004.html
* NEWS: Document this.
* src/create.c (write_gnu_long_link): If --numeric-owner,
leave the user and group empty in a private header. Cache the
names for 0.
* scripts/backup.in: Check exit code of MT_BEGIN
* scripts/backup.sh.in (backup_host): Return meaningful code
in case of remote execution. See the comment, though.
* src/common.h (name_more_files): New proto.
(files_from_option): Remove.
* src/names.c (name_more_files): New file.
(names_options): Fix declaration of the
add-file option.
(names_parse_opt): Handle --add-file.
* src/tar.c (struct tar_args): Remove the input_files member.
Change all uses: use name_more_files() instead.
* tests/Makefile.am: Add new test.
* tests/add-file.at: New testcase.
* tests/testsuite.at: Add new test.
Pavel Raiskup [Thu, 30 Jun 2016 14:17:29 +0000 (16:17 +0200)]
sparse: fix pax extraction for unicode filenames
Make sure that 'GNU.sparse.name' header has higher priority than
(for sparse-purposes artificially modified) 'path' pax header.
Historically, the 'GNU.sparse.name' header comes before 'path';
this caused that modified 'path' header won and that is not what
we want in sparse "capable" tar implementation.
* src/tar.h (tar_stat_info): New argument sparse_name_done.
* src/xheader.c (raw_path_decoder): Move here the unconditional
code from path_decoder.
(path_decoder): Apply raw_path_decoder only if sparse_path_decoder
was not yet called.
(sparse_path_decoder): New wrapper around raw_path_decoder.
* tests/sparse07.at: New testcase.
* tests/testsuite.at: Mention new testcase.
* tests/Makefile.am: Likewise.
Ian McLeod [Mon, 30 May 2016 22:11:35 +0000 (17:11 -0500)]
Bugfix - fix xattr exclude/include for archive create
This makes archive create behavior consistent with the
documentation. Without this change xattr include/exclude options
are accepted when creating an archive but are silently ignored.
* src/xattrs.c (xattrs_xattrs_get): Apply exclude/include mask
when fetching extended attributes
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
Report positional options that were used but had no effect during archive creation
* src/names.c (file_selection_option)
(file_selection_option_name): New functions.
(unconsumed_option_push, unconsumed_option_free)
(unconsumed_option_report): New functions.
(name_list_advance): Maintain a list
of eventually unconsumed options during archive creation.
Report unconsumed options, if any.
* tests/positional01.at: New test case.
* tests/positional02.at: New test case.
* tests/positional03.at: New test case.
* tests/Makefile.am: Add new test cases.
* tests/testsuite.at: Likewise.
* NEWS: Document the changes.
* configure.ac: Version 1.29.90
* doc/tar.texi: Document the changes.
Fix argument handling when running external commands.
* src/system.c (xexec): Use sh -c to run the command. This fixed
bug introduced by 7b5e80396 (tar 1.27)
* doc/tar.texi: Fix checkpoint examples: (1) $TAR_FILENAME
is not available when creating archive and (2) --checkpoint
can't be used as abbreviation of --checkpoint-action
The auxiliary utility ttyemu proved to be unreliable. Given existing
differences between pty implementations and termios ioctls on various
platforms, writing it in a portable way requires effort disproportional
to its actual purpose.
Pavel Raiskup [Tue, 5 Apr 2016 14:53:47 +0000 (17:53 +0300)]
xattrs: fix build on Darwin
Be careful to define HAVE_XATTRS when not all needed xattr-related
functions are properly defined either in libc or libattr.
Reported independently by Denis Excoffier and Dominyk Tille.
* acinclude.m4 (TAR_HEADERS_ATTR_XATTR_H): Check for each xattr
function separately. Don't AC_CHECK_LIB (LIBS is filled by
AC_SEARCH_LIBS when necessary).
* src/Makefile.am: The LDADD -lattr was redundant.
Jeremy Bobbio [Thu, 24 Mar 2016 05:11:28 +0000 (07:11 +0200)]
New option --clamp-mtime
The new `--clamp-mtime` option will change the behavior of `--mtime` to only
use the time specified if the file mtime is newer than the given time.
The `--clamp-mtime` option can only be used together with `--mtime`.
Typical use case is to make builds reproducible: to loose less
information, it's better to keep the original date of an archive, except for
files modified during the build process. In that case, using a reference
(and thus reproducible) timestamps for the latter is good enough. See
<https://wiki.debian.org/ReproducibleBuilds> for more information.
Patch submitted by Jeremy Bobbio and
Daniel Kahn Gillmor <dkg@fifthhorseman.net>
* src/common.h (set_mtime_option_mode): New enum
(set_mtime_option): Change type to enum set_mtime_option_mode.
(NEWER_OPTION_INITIALIZED): Rename to NEWER_OPTION_INITIALIZED.
* src/create.c (start_header): Set mtime depending on set_mtime_option.
* src/tar.c (options,parse_opt): New option --clamp-mtime
(decode_options): Initialize mtime_option
* tests/time02.at: New testcase.
* tests/Makefile.am: Add new testcase
* tests/testsuite.at: Likewise.
* tests/sparse06.at: Don't use timeout: depending on the filesystem
mounted, current LA and lots of other factors, creation of archive can
take much more time than the expected 2 seconds.
* paxutils: Update.
* src/unlink.c (flush_deferred_unlinks): OpenSolaris sets EEXIST
instead of ENOTEMPTY if trying to remove a non-empty directory.
* tests/numeric.at: Avoid using awk -v: some older awks don't support
that option. Also fix environment variable usage.
* tests/onetop05.at: Skip test if unable to set initial directory
mode bits.
* tests/sparse06.at: Use --quiet option.
Fix appending to archive with changed blocking factor.
* src/buffer.c (flush_archive): If previous reading attempt resulted
in short read, correctly use the remaining record space.
(backspace_output): Fix position calculation (still has to be
improved).
* tests/append05.at: New test case.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
* doc/tar.texi: Document position-sensitive options in a
separate subsection.
* src/names.c (names_argp,names_argp_children): Explicitly initialize
all members.
Filename-selection options are --wildcards, --recursive, etc. (see
names.c for a complete list). These options are position-sensitive,
i.e. each such option affects all filenames and filename-selection
options that appear after it until eventually cancelled by a
corresponding counterpart option.
These options can appear in "file-from" file lists, which means that
they cannot be handled right away, but instead should be put on
the "name_elt" list and processed sequentionally, as file name arguments
are.
* src/common.h (warn_regex_usage): Remove.
(name_add_name): Change signature.
(name_add_dir, name_add_file): Remove prototypes.
* src/names.c (name_add_option, name_add_dir)
(name_add_file): Static functions.
(names_options, is_file_selection_option, names_parse_opt): Static functions.
(names_argp_children): New variable.
(NELT_NAME, NELT_CHDIR)
(NELT_FILE, NELT_NOOP): Redefine as enum nelt_type.
(NELT_FMASK): Remove.
(NELT_OPTION): New constant.
(name_elt) <type>: Change type.
<v.opt>: New member.
(name_elt_alloc_matflags): Remove.
(name_add_name): Take one argument.
(name_add_option): New static function.
(name_add_file): Take one argument.
(read_next_name): Use filename_terminator and
verbatim_files_from_option to initialize file.term and file.verbatim.
* src/tar.c: Move handling of filename-selection options to names.c
Pavel Raiskup [Thu, 10 Dec 2015 12:55:34 +0000 (14:55 +0200)]
better test --{,no-}recursion options
* tests/recurs02.at: Also test --list mode.
* tests/T-recurse.at: New test case. Test that -T option works
correctly together with --{,no-}recursion.
* tests/Makefile.am: Mention new test T-recurse.at.
* tests/testsuite.at: Likewise.
Pavel Raiskup [Sun, 6 Dec 2015 19:18:51 +0000 (21:18 +0200)]
numeric-owner: print big UID/GID correctly
* src/list.c (simple_print_header): Do not parse ustar header
for UID/GID again (tar_stat_info has already been correctly
filled with respect to possible uid/gid extended headers).
* tests/numeric.at: New testcase for --numeric-owner option.
* tests/Makefile.am: Add new testcase.
* tests/testsuite.at: Likewise.
Use SEEK_HOLE/SEEK_DATA feature of lseek on systems that support
it. This can make archiving of sparse files much faster.
Implement the --hole-detection option to allow users to select
hole-detection method.
* src/common.h (hole_detection_method): New enum.
(hole_detection): New global.
* src/sparse.c (sparse_scan_file_wholesparse): New function as a
method for detecting sparse files without any data.
(sparse_scan_file_raw): Rename from sparse_scan_file; with edits.
(sparse_scan_file_seek): New function.
(sparse_scan_file): Reimplement function.
* src/tar.c: New option --hole-detection
* tests/checkseekhole.c: New file.
* tests/.gitignore: Mention two test binaries.
* tests/Makefile.am: Add new tests.
* tests/testsuite.at (AT_SEEKHOLE_PREREQ): New macro.
Include sparse06.at.
* tests/sparse06.at: New test case.
* tests/sparse02.at: Force raw hole-detection method.
* tests/sparsemv.at: Likewise.
* tests/sparsemvp.at: Likewise.
* src/misc.c (write_fatal_details): Move to buffer.c
* src/buffer.c (write_fatal_details): Close the archive and wait for
the compressor program to terminate in order to catch eventual execution
errors.
* src/system.c (sys_child_open_for_compress): Ignore SIGPIPE so tar will
not silently terminate when unable to write to the compressor.
* tests/comperr.at: New file.
* tests/Makefile.am: Add comperr.at
* tests/testsuite.at: Include comperr.at
When extracting an archive that contains './' with the --one-top-level option,
the mode and ownership of '.' would be incorrectly applied to the current
working directory, instead of the requested top-level directory.
* src/list.c (enforce_one_top_level): Map '.' to the top-level
directory.
* tests/Makefile.am: Add onetop05.at
* tests/testsuite.at: Include onetop05.at.
* tests/onetop05.at: New file.
* tests/onetop01.at: Fix keywords.
* tests/onetop02.at: Likewise.
* tests/onetop03.at: Likewise.
* tests/onetop04.at: Likewise.
projects / thirdparty / tar.git / log
