xz: Update to version 5.2.8

- Update from version 5.2.5 to 5.2.8
- Update of rootfile
- Remove xzgrep-ZDI-CAN-16587 patch as the contents are now integrated into the source
   tarball and with an improved quicker method - see changelog below.
- Changelog
	5.2.8 (2022-11-13)
	    * xz:
	        - If xz cannot remove an input file when it should, this
	          is now treated as a warning (exit status 2) instead of
	          an error (exit status 1). This matches GNU gzip and it
	          is more logical as at that point the output file has
	          already been successfully closed.
	        - Fix handling of .xz files with an unsupported check type.
	          Previously such printed a warning message but then xz
	          behaved as if an error had occurred (didn't decompress,
	          exit status 1). Now a warning is printed, decompression
	          is done anyway, and exit status is 2. This used to work
	          slightly before 5.0.0. In practice this bug matters only
	          if xz has been built with some check types disabled. As
	          instructed in PACKAGERS, such builds should be done in
	          special situations only.
	        - Fix "xz -dc --single-stream tests/files/good-0-empty.xz"
	          which failed with "Internal error (bug)". That is,
	          --single-stream was broken if the first .xz stream in
	          the input file didn't contain any uncompressed data.
	        - Fix displaying file sizes in the progress indicator when
	          working in passthru mode and there are multiple input files.
	          Just like "gzip -cdf", "xz -cdf" works like "cat" when the
	          input file isn't a supported compressed file format. In
	          this case the file size counters weren't reset between
	          files so with multiple input files the progress indicator
	          displayed an incorrect (too large) value.
	    * liblzma:
	        - API docs in lzma/container.h:
	            * Update the list of decoder flags in the decoder
	              function docs.
	            * Explain LZMA_CONCATENATED behavior with .lzma files
	              in lzma_auto_decoder() docs.
	        - OpenBSD: Use HW_NCPUONLINE to detect the number of
	          available hardware threads in lzma_physmem().
	        - Fix use of wrong macro to detect x86 SSE2 support.
	          __SSE2_MATH__ was used with GCC/Clang but the correct
	          one is __SSE2__. The first one means that SSE2 is used
	          for floating point math which is irrelevant here.
	          The affected SSE2 code isn't used on x86-64 so this affects
	          only 32-bit x86 builds that use -msse2 without -mfpmath=sse
	          (there is no runtime detection for SSE2). It improves LZMA
	          compression speed (not decompression).
	        - Fix the build with Intel C compiler 2021 (ICC, not ICX)
	          on Linux. It defines __GNUC__ to 10 but doesn't support
	          the __symver__ attribute introduced in GCC 10.
	    * Scripts: Ignore warnings from xz by using --quiet --no-warn.
	      This is needed if the input .xz files use an unsupported
	      check type.
	    * Translations:
	        - Updated Croatian and Turkish translations.
	        - One new translations wasn't included because it needed
	          technical fixes. It will be in upcoming 5.4.0. No new
	          translations will be added to the 5.2.x branch anymore.
	        - Renamed the French man page translation file from
	          fr_FR.po to fr.po and thus also its install directory
	          (like /usr/share/man/fr_FR -> .../fr).
	        - Man page translations for upcoming 5.4.0 are now handled
	          in the Translation Project.
	    * Update doc/faq.txt a little so it's less out-of-date.
	5.2.7 (2022-09-30)
	    * liblzma:
	        - Made lzma_filters_copy() to never modify the destination
	          array if an error occurs. lzma_stream_encoder() and
	          lzma_stream_encoder_mt() already assumed this. Before this
	          change, if a tiny memory allocation in lzma_filters_copy()
	          failed it would lead to a crash (invalid free() or invalid
	          memory reads) in the cleanup paths of these two encoder
	          initialization functions.
	        - Added missing integer overflow check to lzma_index_append().
	          This affects xz --list and other applications that decode
	          the Index field from .xz files using lzma_index_decoder().
	          Normal decompression of .xz files doesn't call this code
	          and thus most applications using liblzma aren't affected
	          by this bug.
	        - Single-threaded .xz decoder (lzma_stream_decoder()): If
	          lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible
	          to use lzma_memlimit_set() to increase the limit and continue
	          decoding. This was supposed to work from the beginning
	          but there was a bug. With other decoders (.lzma or
	          threaded .xz decoder) this already worked correctly.
	        - Fixed accumulation of integrity check type statistics in
	          lzma_index_cat(). This bug made lzma_index_checks() return
	          only the type of the integrity check of the last Stream
	          when multiple lzma_indexes were concatenated. Most
	          applications don't use these APIs but in xz it made
	          xz --list not list all check types from concatenated .xz
	          files. In xz --list --verbose only the per-file "Check:"
	          lines were affected and in xz --robot --list only the "file"
	          line was affected.
	        - Added ABI compatibility with executables that were linked
	          against liblzma in RHEL/CentOS 7 or other liblzma builds
	          that had copied the problematic patch from RHEL/CentOS 7
	          (xz-5.2.2-compat-libs.patch). For the details, see the
	          comment at the top of src/liblzma/validate_map.sh.
	          WARNING: This uses __symver__ attribute with GCC >= 10.
	          In other cases the traditional __asm__(".symver ...")
	          is used. Using link-time optimization (LTO, -flto) with
	          GCC versions older than 10 can silently result in
	          broken liblzma.so.5 (incorrect symbol versions)! If you
	          want to use -flto with GCC, you must use GCC >= 10.
	          LTO with Clang seems to work even with the traditional
	          __asm__(".symver ...") method.
	    * xzgrep: Fixed compatibility with old shells that break if
	      comments inside command substitutions have apostrophes (').
	      This problem was introduced in 5.2.6.
	    * Build systems:
	        - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX
	        - Windows: Fixed liblzma.dll build with Visual Studio project
	          files. It broke in 5.2.6 due to a change that was made to
	          improve CMake support.
	        - Windows: Building liblzma with UNICODE defined should now
	          work.
	        - CMake files are now actually included in the release tarball.
	          They should have been in 5.2.5 already.
	        - Minor CMake fixes and improvements.
	    * Added a new translation: Turkish
	5.2.6 (2022-08-12)
	    * xz:
	        - The --keep option now accepts symlinks, hardlinks, and
	          setuid, setgid, and sticky files. Previously this required
	          using --force.
	        - When copying metadata from the source file to the destination
	          file, don't try to set the group (GID) if it is already set
	          correctly. This avoids a failure on OpenBSD (and possibly on
	          a few other OSes) where files may get created so that their
	          group doesn't belong to the user, and fchown(2) can fail even
	          if it needs to do nothing.
	        - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on
	          MIPS32 because on MIPS32 userspace processes are limited
	          to 2 GiB of address space.
	    * liblzma:
	        - Fixed a missing error-check in the threaded encoder. If a
	          small memory allocation fails, a .xz file with an invalid
	          Index field would be created. Decompressing such a file would
	          produce the correct output but result in an error at the end.
	          Thus this is a "mild" data corruption bug. Note that while
	          a failed memory allocation can trigger the bug, it cannot
	          cause invalid memory access.
	        - The decoder for .lzma files now supports files that have
	          uncompressed size stored in the header and still use the
	          end of payload marker (end of stream marker) at the end
	          of the LZMA stream. Such files are rare but, according to
	          the documentation in LZMA SDK, they are valid.
	          doc/lzma-file-format.txt was updated too.
	        - Improved 32-bit x86 assembly files:
	            * Support Intel Control-flow Enforcement Technology (CET)
	            * Use non-executable stack on FreeBSD.
	        - Visual Studio: Use non-standard _MSVC_LANG to detect C++
	          standard version in the lzma.h API header. It's used to
	          detect when "noexcept" can be used.
	    * xzgrep:
	        - Fixed arbitrary command injection via a malicious filename
	          (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for
	          this was released to the public on 2022-04-07. A slight
	          robustness improvement has been made since then and, if
	          using GNU or *BSD grep, a new faster method is now used
	          that doesn't use the old sed-based construct at all. This
	          also fixes bad output with GNU grep >= 3.5 (2020-09-27)
	          when xzgrepping binary files.
	          This vulnerability was discovered by:
	          cleemy desu wayo working with Trend Micro Zero Day Initiative
	        - Fixed detection of corrupt .bz2 files.
	        - Improved error handling to fix exit status in some situations
	          and to fix handling of signals: in some situations a signal
	          didn't make xzgrep exit when it clearly should have. It's
	          possible that the signal handling still isn't quite perfect
	          but hopefully it's good enough.
	        - Documented exit statuses on the man page.
	        - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead
	          of the deprecated egrep and fgrep commands.
	        - Fixed parsing of the options -E, -F, -G, -P, and -X. The
	          problem occurred when multiple options were specied in
	          a single argument, for example,
	              echo foo | xzgrep -Fe foo
	          treated foo as a filename because -Fe wasn't correctly
	          split into -F -e.
	        - Added zstd support.
	    * xzdiff/xzcmp:
	        - Fixed wrong exit status. Exit status could be 2 when the
	          correct value is 1.
	        - Documented on the man page that exit status of 2 is used
	          for decompression errors.
	        - Added zstd support.
	    * xzless:
	        - Fix less(1) version detection. It failed if the version number
	          from "less -V" contained a dot.
	    * Translations:
	        - Added new translations: Catalan, Croatian, Esperanto,
	          Korean, Portuguese, Romanian, Serbian, Spanish, Swedish,
	          and Ukrainian
	        - Updated the Brazilian Portuguese translation.
	        - Added French man page translation. This and the existing
	          German translation aren't complete anymore because the
	          English man pages got a few updates and the translators
	          weren't reached so that they could update their work.
	    * Build systems:
	        - Windows: Fix building of resource files when config.h isn't
	          used. CMake + Visual Studio can now build liblzma.dll.
	        - Various fixes to the CMake support. Building static or shared
	          liblzma should work fine in most cases. In contrast, building
	          the command line tools with CMake is still clearly incomplete
	          and experimental and should be used for testing only.

Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
Reviewed-by: Michael Tremer <michael.tremer@ipfire.org>

diff --git a/config/rootfiles/common/xz b/config/rootfiles/common/xz
index c7bd0b3022fab428db23082cba142918350b3469..d2f1d44ccb6d72b87fb592e0ade15286fe991b51 100644 (file)
--- a/config/rootfiles/common/xz
+++ b/config/rootfiles/common/xz
@@ -41,7 +41,7 @@ usr/bin/xzmore
 #usr/lib/liblzma.la
 usr/lib/liblzma.so
 usr/lib/liblzma.so.5
-usr/lib/liblzma.so.5.2.5
+usr/lib/liblzma.so.5.2.8
 #usr/lib/pkgconfig/liblzma.pc
 #usr/share/doc/xz
 #usr/share/doc/xz/AUTHORS
@@ -65,15 +65,26 @@ usr/lib/liblzma.so.5.2.5
 #usr/share/doc/xz/history.txt
 #usr/share/doc/xz/lzma-file-format.txt
 #usr/share/doc/xz/xz-file-format.txt
+#usr/share/locale/ca/LC_MESSAGES/xz.mo
 #usr/share/locale/cs/LC_MESSAGES/xz.mo
 #usr/share/locale/da/LC_MESSAGES/xz.mo
 #usr/share/locale/de/LC_MESSAGES/xz.mo
+#usr/share/locale/eo/LC_MESSAGES/xz.mo
+#usr/share/locale/es/LC_MESSAGES/xz.mo
 #usr/share/locale/fi/LC_MESSAGES/xz.mo
 #usr/share/locale/fr/LC_MESSAGES/xz.mo
+#usr/share/locale/hr/LC_MESSAGES/xz.mo
 #usr/share/locale/hu/LC_MESSAGES/xz.mo
 #usr/share/locale/it/LC_MESSAGES/xz.mo
+#usr/share/locale/ko/LC_MESSAGES/xz.mo
 #usr/share/locale/pl/LC_MESSAGES/xz.mo
+#usr/share/locale/pt/LC_MESSAGES/xz.mo
 #usr/share/locale/pt_BR/LC_MESSAGES/xz.mo
+#usr/share/locale/ro/LC_MESSAGES/xz.mo
+#usr/share/locale/sr/LC_MESSAGES/xz.mo
+#usr/share/locale/sv/LC_MESSAGES/xz.mo
+#usr/share/locale/tr/LC_MESSAGES/xz.mo
+#usr/share/locale/uk/LC_MESSAGES/xz.mo
 #usr/share/locale/vi/LC_MESSAGES/xz.mo
 #usr/share/locale/zh_CN/LC_MESSAGES/xz.mo
 #usr/share/locale/zh_TW/LC_MESSAGES/xz.mo
@@ -82,9 +93,6 @@ usr/lib/liblzma.so.5.2.5
 #usr/share/man/de/man1/lzcat.1
 #usr/share/man/de/man1/lzcmp.1
 #usr/share/man/de/man1/lzdiff.1
-#usr/share/man/de/man1/lzegrep.1
-#usr/share/man/de/man1/lzfgrep.1
-#usr/share/man/de/man1/lzgrep.1
 #usr/share/man/de/man1/lzless.1
 #usr/share/man/de/man1/lzma.1
 #usr/share/man/de/man1/lzmadec.1
@@ -96,11 +104,26 @@ usr/lib/liblzma.so.5.2.5
 #usr/share/man/de/man1/xzcmp.1
 #usr/share/man/de/man1/xzdec.1
 #usr/share/man/de/man1/xzdiff.1
-#usr/share/man/de/man1/xzegrep.1
-#usr/share/man/de/man1/xzfgrep.1
-#usr/share/man/de/man1/xzgrep.1
 #usr/share/man/de/man1/xzless.1
 #usr/share/man/de/man1/xzmore.1
+#usr/share/man/fr
+#usr/share/man/fr/man1
+#usr/share/man/fr/man1/lzcat.1
+#usr/share/man/fr/man1/lzcmp.1
+#usr/share/man/fr/man1/lzdiff.1
+#usr/share/man/fr/man1/lzless.1
+#usr/share/man/fr/man1/lzma.1
+#usr/share/man/fr/man1/lzmadec.1
+#usr/share/man/fr/man1/lzmore.1
+#usr/share/man/fr/man1/unlzma.1
+#usr/share/man/fr/man1/unxz.1
+#usr/share/man/fr/man1/xz.1
+#usr/share/man/fr/man1/xzcat.1
+#usr/share/man/fr/man1/xzcmp.1
+#usr/share/man/fr/man1/xzdec.1
+#usr/share/man/fr/man1/xzdiff.1
+#usr/share/man/fr/man1/xzless.1
+#usr/share/man/fr/man1/xzmore.1
 #usr/share/man/man1/lzcat.1
 #usr/share/man/man1/lzcmp.1
 #usr/share/man/man1/lzdiff.1

diff --git a/lfs/xz b/lfs/xz
index 9345df95427aeaa1d1eb8fe9985bcaf4e3217aa3..83a724e1a3177d632b806995c4b0dc3e2ccb25aa 100644 (file)
--- a/lfs/xz
+++ b/lfs/xz
@@ -24,7 +24,7 @@
 
 include Config
 
-VER        = 5.2.5
+VER        = 5.2.8
 
 THISAPP    = xz-$(VER)
 DL_FILE    = $(THISAPP).tar.xz
@@ -45,7 +45,7 @@ objects = $(DL_FILE)
 
 $(DL_FILE) = $(DL_FROM)/$(DL_FILE)
 
-$(DL_FILE)_BLAKE2 = 9b9b58e33722ecfe799bb50f3ffe4e86386f734ab4468eb54ff92771ddf899302d1ffa4d88bdb0de351fc3eab8a6ea103f00d7e79f33db879fe22b2e1a7e62db
+$(DL_FILE)_BLAKE2 = 44d1ddd783b2527f3b17481fc277b671808eb5639c10d31bfaca9fd29ac4413628654ecb9e207955a9477c83eb30f61cf5607cd9a49dd71732707731e4444ace
 
 install : $(TARGET)
 
@@ -75,7 +75,6 @@ $(subst %,%_BLAKE2,$(objects)) :
 $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
        @$(PREBUILD)
        @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
-       cd $(DIR_APP) && patch -p1 -i $(DIR_SRC)/src/patches/xzgrep-ZDI-CAN-16587.patch
        cd $(DIR_APP) && ./configure --prefix=$(PREFIX)
        cd $(DIR_APP) && make $(MAKETUNING)
        cd $(DIR_APP) && make install

diff --git a/src/patches/xzgrep-ZDI-CAN-16587.patch b/src/patches/xzgrep-ZDI-CAN-16587.patch
deleted file mode 100644 (file)
index 406ded5..0000000
--- a/src/patches/xzgrep-ZDI-CAN-16587.patch
+++ /dev/null
@@ -1,94 +0,0 @@
-From 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 Mon Sep 17 00:00:00 2001
-From: Lasse Collin <lasse.collin@tukaani.org>
-Date: Tue, 29 Mar 2022 19:19:12 +0300
-Subject: [PATCH] xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587).
-
-Malicious filenames can make xzgrep to write to arbitrary files
-or (with a GNU sed extension) lead to arbitrary code execution.
-
-xzgrep from XZ Utils versions up to and including 5.2.5 are
-affected. 5.3.1alpha and 5.3.2alpha are affected as well.
-This patch works for all of them.
-
-This bug was inherited from gzip's zgrep. gzip 1.12 includes
-a fix for zgrep.
-
-The issue with the old sed script is that with multiple newlines,
-the N-command will read the second line of input, then the
-s-commands will be skipped because it's not the end of the
-file yet, then a new sed cycle starts and the pattern space
-is printed and emptied. So only the last line or two get escaped.
-
-One way to fix this would be to read all lines into the pattern
-space first. However, the included fix is even simpler: All lines
-except the last line get a backslash appended at the end. To ensure
-that shell command substitution doesn't eat a possible trailing
-newline, a colon is appended to the filename before escaping.
-The colon is later used to separate the filename from the grep
-output so it is fine to add it here instead of a few lines later.
-
-The old code also wasn't POSIX compliant as it used \n in the
-replacement section of the s-command. Using \<newline> is the
-POSIX compatible method.
-
-LC_ALL=C was added to the two critical sed commands. POSIX sed
-manual recommends it when using sed to manipulate pathnames
-because in other locales invalid multibyte sequences might
-cause issues with some sed implementations. In case of GNU sed,
-these particular sed scripts wouldn't have such problems but some
-other scripts could have, see:
-
-    info '(sed)Locale Considerations'
-
-This vulnerability was discovered by:
-cleemy desu wayo working with Trend Micro Zero Day Initiative
-
-Thanks to Jim Meyering and Paul Eggert discussing the different
-ways to fix this and for coordinating the patch release schedule
-with gzip.
----
- src/scripts/xzgrep.in | 20 ++++++++++++--------
- 1 file changed, 12 insertions(+), 8 deletions(-)
-
-diff --git a/src/scripts/xzgrep.in b/src/scripts/xzgrep.in
-index b180936..e5186ba 100644
---- a/src/scripts/xzgrep.in
-+++ b/src/scripts/xzgrep.in
-@@ -180,22 +180,26 @@ for i; do
-          { test $# -eq 1 || test $no_filename -eq 1; }; then
-       eval "$grep"
-     else
-+      # Append a colon so that the last character will never be a newline
-+      # which would otherwise get lost in shell command substitution.
-+      i="$i:"
-+
-+      # Escape & \ | and newlines only if such characters are present
-+      # (speed optimization).
-       case $i in
-       (*'
- '* | *'&'* | *'\'* | *'|'*)
--        i=$(printf '%s\n' "$i" |
--            sed '
--              $!N
--              $s/[&\|]/\\&/g
--              $s/\n/\\n/g
--            ');;
-+        i=$(printf '%s\n' "$i" | LC_ALL=C sed 's/[&\|]/\\&/g; $!s/$/\\/');;
-       esac
--      sed_script="s|^|$i:|"
-+
-+      # $i already ends with a colon so don't add it here.
-+      sed_script="s|^|$i|"
- 
-       # Fail if grep or sed fails.
-       r=$(
-         exec 4>&1
--        (eval "$grep" 4>&-; echo $? >&4) 3>&- | sed "$sed_script" >&3 4>&-
-+        (eval "$grep" 4>&-; echo $? >&4) 3>&- |
-+            LC_ALL=C sed "$sed_script" >&3 4>&-
-       ) || r=2
-       exit $r
-     fi >&3 5>&-
--- 
-2.35.1
-