]>
Commit | Line | Data |
---|---|---|
6e1326fc | 1 | |
d91a84b5 LC |
2 | XZ Utils Release Notes |
3 | ====================== | |
4 | ||
eeedd4d0 LC |
5 | 5.5.2beta (2024-02-14) |
6 | ||
7 | * Licensing change: The core components are now under the | |
8 | BSD Zero Clause License (0BSD). In XZ Utils 5.4.6 and older | |
9 | and 5.5.1alpha these components are in the public domain and | |
10 | obviously remain so; the change affects the new releases only. | |
11 | ||
12 | 0BSD is an extremely permissive license which doesn't require | |
13 | retaining or reproducing copyright or license notices when | |
14 | distributing the code, thus in practice there is extremely | |
15 | little difference to public domain. | |
16 | ||
17 | * liblzma | |
18 | ||
19 | - Significant speed optimizations to the LZMA decoder were | |
20 | made. There are now three variants that can be chosen at | |
21 | build time: | |
22 | ||
23 | * Basic C version: This is a few percent faster than | |
24 | 5.4.x due to some new optimizations. | |
25 | ||
26 | * Branchless C: This is currently the default on platforms | |
27 | for which there is no assembly code. This should be a few | |
28 | percent faster than the basic C version. | |
29 | ||
30 | * x86-64 inline assembly. This works with GCC and Clang. | |
31 | ||
fb5f6aaf | 32 | The default choice can currently be overridden by setting |
eeedd4d0 LC |
33 | LZMA_RANGE_DECODER_CONFIG in CPPFLAGS: 0 means the basic |
34 | version and 3 means that branchless C version. | |
35 | ||
36 | - Optimized the CRC32 calculation on ARM64 platforms using the | |
37 | CRC32 instructions. The instructions are optional in ARMv8.0 | |
38 | and are required in ARMv8.1 and later. Runtime detection for | |
39 | the instruction is used on GNU/Linux, FreeBSD, Windows, and | |
40 | macOS. If the compiler flags indicate unconditional CRC32 | |
41 | instruction support (+crc) then the generic version is not | |
42 | built. | |
43 | ||
44 | * Added lz4 support to xzdiff/xzcmp and xzgrep. | |
45 | ||
46 | * Man pages of xzdiff/xzcmp, xzgrep, and xzmore were rewritten | |
47 | to simplify licensing of the man page translations. | |
48 | ||
49 | * Translations: | |
50 | ||
51 | - Updated Chinese (simplified), German, Korean, Polish, | |
52 | Romanian, Spanish, Swedish, and Ukrainian translations. | |
53 | ||
54 | - Updated German, Korean, Romanian, and Ukrainian man page | |
55 | translations. | |
56 | ||
57 | * Small improvements to the tests. | |
58 | ||
0b8cefa1 LC |
59 | * Added doc/examples/11_file_info.c. It was added to the Git |
60 | repository in 2017 but forgotten to be added into distribution | |
61 | tarballs. | |
eeedd4d0 LC |
62 | |
63 | * Removed doc/examples_old. These were from 2012. | |
64 | ||
0b8cefa1 LC |
65 | * Removed the macos/build.sh script. It had not been updated |
66 | since 2013. | |
67 | ||
eeedd4d0 | 68 | |
c7a7ae15 JT |
69 | 5.5.1alpha (2024-01-26) |
70 | ||
71 | * Added a new filter for RISC-V binaries. The filter can be used | |
72 | for 32-bit and 64-bit binaries with either little or big | |
73 | endianness. In liblzma, the Filter ID is LZMA_FILTER_RISCV (0x0B) | |
74 | and the xz option is --riscv. liblzma filter string syntax | |
75 | recognizes this filter as "riscv". | |
76 | ||
77 | * liblzma: | |
78 | ||
79 | - Added lzma_mt_block_size() to recommend a Block size for | |
80 | multithreaded encoding | |
81 | ||
82 | - Added CLMUL-based CRC32 on x86-64 and E2K with runtime | |
83 | processor detection. Similar to CRC64, on 32-bit x86 it | |
84 | isn't available unless --disable-assembler is used. | |
85 | ||
86 | - Implemented GNU indirect function (IFUNC) as a runtime | |
87 | function dispatching method for CRC32 and CRC64 fast | |
88 | implementations on x86. Only GNU/Linux (glibc) and FreeBSD | |
89 | builds will use IFUNC, unless --enable-ifunc is specified to | |
90 | configure. | |
91 | ||
92 | - Added definitions of mask values like | |
93 | LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>. | |
94 | ||
95 | - The XZ logo is now included in the Doxygen generated | |
96 | documentation. It is licensed under Creative Commons | |
97 | Attribution-ShareAlike 4.0. | |
98 | ||
99 | * xz: | |
100 | ||
101 | - Multithreaded mode is now the default. This improves | |
102 | compression speed and creates .xz files that can be | |
103 | decompressed multithreaded at the cost of increased memory | |
104 | usage and slightly worse compression ratio. | |
105 | ||
106 | - Added new command line option --filters to set the filter | |
107 | chain using liblzma filter string syntax. | |
108 | ||
109 | - Added new command line options --filters1 ... --filters9 to | |
110 | set additional filter chains using liblzma filter string | |
111 | syntax. The --block-list option now allows specifying filter | |
112 | chains that were set using these new options. | |
113 | ||
114 | - Added support for Linux Landlock as a sandboxing method. | |
115 | ||
116 | - xzdec now supports pledge(2), Capsicum, and Linux Landlock as | |
117 | sandboxing methods. | |
118 | ||
119 | - Progress indicator time stats remain accurate after pausing | |
120 | xz with SIGTSTP. | |
121 | ||
122 | - Ported xz and xzdec to Windows MSVC. Visual Studio 2015 or | |
123 | later is required. | |
124 | ||
125 | * CMake Build: | |
126 | ||
127 | - Supports pledge(2), Capsicum, and Linux Landlock sandboxing | |
128 | methods. | |
129 | ||
130 | - Replacement functions for getopt_long() are used on platforms | |
131 | that do not have it. | |
132 | ||
133 | * Enabled unaligned access by default on PowerPC64LE and on RISC-V | |
134 | targets that define __riscv_misaligned_fast. | |
135 | ||
136 | * Tests: | |
137 | ||
138 | - Added two new fuzz targets to OSS-Fuzz. | |
139 | ||
140 | - Implemented Continuous Integration (CI) testing using | |
141 | GitHub Actions. | |
142 | ||
143 | * Changed quoting style from `...' to '...' in all messages, | |
144 | scripts, and documentation. | |
145 | ||
146 | * Added basic Codespell support to help catch typo errors. | |
147 | ||
148 | ||
0ef8192e JT |
149 | 5.4.6 (2024-01-26) |
150 | ||
151 | * Fixed a bug involving internal function pointers in liblzma not | |
152 | being initialized to NULL. The bug can only be triggered if | |
153 | lzma_filters_update() is called on a LZMA1 encoder, so it does | |
154 | not affect xz or any application known to us that uses liblzma. | |
155 | ||
156 | * xz: | |
157 | ||
158 | - Fixed a regression introduced in 5.4.2 that caused encoding | |
159 | in the raw format to unnecessarily fail if --suffix was not | |
160 | used. For instance, the following command no longer reports | |
161 | that --suffix must be used: | |
162 | ||
163 | echo foo | xz --format=raw --lzma2 | wc -c | |
164 | ||
165 | - Fixed an issue on MinGW-w64 builds that prevented reading | |
166 | from or writing to non-terminal character devices like NUL. | |
167 | ||
168 | * Added a new test. | |
169 | ||
170 | ||
5fe14506 JT |
171 | 5.4.5 (2023-11-31) |
172 | ||
173 | * liblzma: | |
174 | ||
175 | - Use __attribute__((__no_sanitize_address__)) to avoid address | |
176 | sanitization with CRC64 CLMUL. It uses 16-byte-aligned reads | |
177 | which can extend past the bounds of the input buffer and | |
178 | inherently trigger address sanitization errors. This isn't | |
179 | a bug. | |
180 | ||
181 | - Fixed an assertion failure that could be triggered by a large | |
182 | unpadded_size argument. It was verified that there was no | |
183 | other bug than the assertion failure. | |
184 | ||
185 | - Fixed a bug that prevented building with Windows Vista | |
186 | threading when __attribute__((__constructor__)) is not | |
187 | supported. | |
188 | ||
189 | * xz now properly handles special files such as "con" or "nul" on | |
190 | Windows. Before this fix, the following wrote "foo" to the | |
191 | console and deleted the input file "con_xz": | |
192 | ||
193 | echo foo | xz > con_xz | |
194 | xz --suffix=_xz --decompress con_xz | |
195 | ||
196 | * Build systems: | |
197 | ||
198 | - Allow builds with Windows win95 threading and small mode when | |
199 | __attribute__((__constructor__)) is supported. | |
200 | ||
201 | - Added a new line to liblzma.pc for MSYS2 (Windows): | |
202 | ||
203 | Cflags.private: -DLZMA_API_STATIC | |
204 | ||
205 | When compiling code that will link against static liblzma, | |
206 | the LZMA_API_STATIC macro needs to be defined on Windows. | |
207 | ||
208 | - CMake specific changes: | |
209 | ||
210 | * Fixed a bug that allowed CLOCK_MONOTONIC to be used even | |
211 | if the check for it failed. | |
212 | ||
213 | * Fixed a bug where configuring CMake multiple times | |
214 | resulted in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC | |
215 | not being set. | |
216 | ||
217 | * Fixed the build with MinGW-w64-based Clang/LLVM 17. | |
218 | llvm-windres now has more accurate GNU windres emulation | |
219 | so the GNU windres workaround from 5.4.1 is needed with | |
220 | llvm-windres version 17 too. | |
221 | ||
222 | * The import library on Windows is now properly named | |
223 | "liblzma.dll.a" instead of "libliblzma.dll.a" | |
224 | ||
225 | * Fixed a bug causing the Ninja Generator to fail on | |
226 | UNIX-like systems. This bug was introduced in 5.4.0. | |
227 | ||
228 | * Added a new option to disable CLMUL CRC64. | |
229 | ||
230 | * A module-definition (.def) file is now created when | |
231 | building liblzma.dll with MinGW-w64. | |
232 | ||
233 | * The pkg-config liblzma.pc file is now installed on all | |
234 | builds except when using MSVC on Windows. | |
235 | ||
236 | * Added large file support by default for platforms that | |
237 | need it to handle files larger than 2 GiB. This includes | |
238 | MinGW-w64, even 64-bit builds. | |
239 | ||
240 | * Small fixes and improvements to the tests. | |
241 | ||
242 | * Updated translations: Chinese (simplified) and Esperanto. | |
243 | ||
244 | ||
9ae4371b JT |
245 | 5.4.4 (2023-08-02) |
246 | ||
247 | * liblzma and xzdec can now build against WASI SDK when threading | |
248 | support is disabled. xz and tests don't build yet. | |
249 | ||
250 | * CMake: | |
251 | ||
252 | - Fixed a bug preventing other projects from including liblzma | |
253 | multiple times using find_package(). | |
254 | ||
255 | - Don't create broken symlinks in Cygwin and MSYS2 unless | |
256 | supported by the environment. This prevented building for the | |
257 | default MSYS2 environment. The problem was introduced in | |
258 | xz 5.4.0. | |
259 | ||
260 | * Documentation: | |
261 | ||
262 | - Small improvements to man pages. | |
263 | ||
264 | - Small improvements and typo fixes for liblzma API | |
265 | documentation. | |
266 | ||
267 | * Tests: | |
268 | ||
269 | - Added a new section to INSTALL to describe basic test usage | |
270 | and address recent questions about building the tests when | |
271 | cross compiling. | |
272 | ||
273 | - Small fixes and improvements to the tests. | |
274 | ||
275 | * Translations: | |
276 | ||
277 | - Fixed a mistake that caused one of the error messages to not | |
278 | be translated. This only affected versions 5.4.2 and 5.4.3. | |
279 | ||
280 | - Updated the Chinese (simplified), Croatian, Esperanto, German, | |
281 | Korean, Polish, Romanian, Spanish, Swedish, Ukrainian, and | |
282 | Vietnamese translations. | |
283 | ||
284 | - Updated the German, Korean, Romanian, and Ukrainian man page | |
285 | translations. | |
286 | ||
287 | ||
c247d06e JT |
288 | 5.4.3 (2023-05-04) |
289 | ||
290 | * All fixes from 5.2.12 | |
291 | ||
292 | * Features in the CMake build can now be disabled as CMake cache | |
293 | variables, similar to the Autotools build. | |
294 | ||
295 | * Minor update to the Croatian translation. | |
296 | ||
297 | ||
f0c580c5 JT |
298 | 5.4.2 (2023-03-18) |
299 | ||
300 | * All fixes from 5.2.11 that were not included in 5.4.1. | |
301 | ||
302 | * If xz is built with support for the Capsicum sandbox but running | |
303 | in an environment that doesn't support Capsicum, xz now runs | |
304 | normally without sandboxing instead of exiting with an error. | |
305 | ||
306 | * liblzma: | |
307 | ||
308 | - Documentation was updated to improve the style, consistency, | |
309 | and completeness of the liblzma API headers. | |
310 | ||
311 | - The Doxygen-generated HTML documentation for the liblzma API | |
312 | header files is now included in the source release and is | |
313 | installed as part of "make install". All JavaScript is | |
314 | removed to simplify license compliance and to reduce the | |
315 | install size. | |
316 | ||
317 | - Fixed a minor bug in lzma_str_from_filters() that produced | |
318 | too many filters in the output string instead of reporting | |
319 | an error if the input array had more than four filters. This | |
320 | bug did not affect xz. | |
321 | ||
322 | * Build systems: | |
323 | ||
324 | - autogen.sh now invokes the doxygen tool via the new wrapper | |
325 | script doxygen/update-doxygen, unless the command line option | |
326 | --no-doxygen is used. | |
327 | ||
328 | - Added microlzma_encoder.c and microlzma_decoder.c to the | |
329 | VS project files for Windows and to the CMake build. These | |
330 | should have been included in 5.3.2alpha. | |
331 | ||
332 | * Tests: | |
333 | ||
334 | - Added a test to the CMake build that was forgotten in the | |
335 | previous release. | |
336 | ||
337 | - Added and refactored a few tests. | |
338 | ||
339 | * Translations: | |
340 | ||
341 | - Updated the Brazilian Portuguese translation. | |
342 | ||
343 | - Added Brazilian Portuguese man page translation. | |
344 | ||
345 | ||
62efd48a JT |
346 | 5.4.1 (2023-01-11) |
347 | ||
348 | * liblzma: | |
349 | ||
350 | - Fixed the return value of lzma_microlzma_encoder() if the | |
351 | LZMA options lc/lp/pb are invalid. Invalid lc/lp/pb options | |
352 | made the function return LZMA_STREAM_END without encoding | |
353 | anything instead of returning LZMA_OPTIONS_ERROR. | |
354 | ||
355 | - Windows / Visual Studio: Workaround a possible compiler bug | |
356 | when targeting 32-bit x86 and compiling the CLMUL version of | |
357 | the CRC64 code. The CLMUL code isn't enabled by the Windows | |
358 | project files but it is in the CMake-based builds. | |
359 | ||
360 | * Build systems: | |
361 | ||
362 | - Windows-specific CMake changes: | |
363 | ||
364 | * Don't try to enable CLMUL CRC64 code if _mm_set_epi64x() | |
365 | isn't available. This fixes CMake-based build with Visual | |
366 | Studio 2013. | |
367 | ||
368 | * Created a workaround for a build failure with windres | |
369 | from GNU binutils. It is used only when the C compiler | |
370 | is GCC (not Clang). The workaround is incompatible | |
371 | with llvm-windres, resulting in "XZx20Utils" instead | |
372 | of "XZ Utils" in the resource file, but without the | |
373 | workaround llvm-windres works correctly. See the | |
374 | comment in CMakeLists.txt for details. | |
375 | ||
376 | * Included the resource files in the xz and xzdec build | |
377 | rules. Building the command line tools is still | |
378 | experimental but possible with MinGW-w64. | |
379 | ||
380 | - Visual Studio: Added stream_decoder_mt.c to the project | |
381 | files. Now the threaded decompressor lzma_stream_decoder_mt() | |
382 | gets built. CMake-based build wasn't affected. | |
383 | ||
384 | - Updated windows/INSTALL-MSVC.txt to mention that CMake-based | |
385 | build is now the preferred method with Visual Studio. The | |
386 | project files will probably be removed after 5.4.x releases. | |
387 | ||
388 | - Changes to #defines in config.h: | |
389 | ||
390 | * HAVE_DECL_CLOCK_MONOTONIC was replaced by | |
391 | HAVE_CLOCK_MONOTONIC. The old macro was always defined | |
392 | in configure-generated config.h to either 0 or 1. The | |
393 | new macro is defined (to 1) only if the declaration of | |
394 | CLOCK_MONOTONIC is available. This matches the way most | |
395 | other config.h macros work and makes things simpler with | |
396 | other build systems. | |
397 | ||
398 | * HAVE_DECL_PROGRAM_INVOCATION_NAME was replaced by | |
399 | HAVE_PROGRAM_INVOCATION_NAME for the same reason. | |
400 | ||
401 | * Tests: | |
402 | ||
403 | - Fixed test script compatibility with ancient /bin/sh | |
404 | versions. Now the five test_compress_* tests should | |
405 | no longer fail on Solaris 10. | |
406 | ||
407 | - Added and refactored a few tests. | |
408 | ||
409 | * Translations: | |
410 | ||
411 | - Updated the Catalan and Esperanto translations. | |
412 | ||
413 | - Added Korean and Ukrainian man page translations. | |
414 | ||
415 | ||
cbbd8445 LC |
416 | 5.4.0 (2022-12-13) |
417 | ||
418 | This bumps the minor version of liblzma because new features were | |
419 | added. The API and ABI are still backward compatible with liblzma | |
420 | 5.2.x and 5.0.x. | |
421 | ||
422 | Since 5.3.5beta: | |
423 | ||
424 | * All fixes from 5.2.10. | |
425 | ||
426 | * The ARM64 filter is now stable. The xz option is now --arm64. | |
427 | Decompression requires XZ Utils 5.4.0. In the future the ARM64 | |
428 | filter will be supported by XZ for Java, XZ Embedded (including | |
429 | the version in Linux), LZMA SDK, and 7-Zip. | |
430 | ||
431 | * Translations: | |
432 | ||
433 | - Updated Catalan, Croatian, German, Romanian, and Turkish | |
434 | translations. | |
435 | ||
436 | - Updated German man page translations. | |
437 | ||
438 | - Added Romanian man page translations. | |
439 | ||
440 | Summary of new features added in the 5.3.x development releases: | |
441 | ||
442 | * liblzma: | |
443 | ||
444 | - Added threaded .xz decompressor lzma_stream_decoder_mt(). | |
445 | It can use multiple threads with .xz files that have multiple | |
446 | Blocks with size information in Block Headers. The threaded | |
447 | encoder in xz has always created such files. | |
448 | ||
449 | Single-threaded encoder cannot store the size information in | |
450 | Block Headers even if one used LZMA_FULL_FLUSH to create | |
451 | multiple Blocks, so this threaded decoder cannot use multiple | |
452 | threads with such files. | |
453 | ||
454 | If there are multiple Streams (concatenated .xz files), one | |
455 | Stream will be decompressed completely before starting the | |
456 | next Stream. | |
457 | ||
458 | - A new decoder flag LZMA_FAIL_FAST was added. It makes the | |
459 | threaded decompressor report errors soon instead of first | |
460 | flushing all pending data before the error location. | |
461 | ||
462 | - New Filter IDs: | |
463 | * LZMA_FILTER_ARM64 is for ARM64 binaries. | |
464 | * LZMA_FILTER_LZMA1EXT is for raw LZMA1 streams that don't | |
465 | necessarily use the end marker. | |
466 | ||
467 | - Added lzma_str_to_filters(), lzma_str_from_filters(), and | |
468 | lzma_str_list_filters() to convert a preset or a filter chain | |
469 | string to a lzma_filter[] and vice versa. These should make | |
470 | it easier to write applications that allow users to specify | |
471 | custom compression options. | |
472 | ||
473 | - Added lzma_filters_free() which can be convenient for freeing | |
474 | the filter options in a filter chain (an array of lzma_filter | |
475 | structures). | |
476 | ||
477 | - lzma_file_info_decoder() to makes it a little easier to get | |
478 | the Index field from .xz files. This helps in getting the | |
479 | uncompressed file size but an easy-to-use random access | |
480 | API is still missing which has existed in XZ for Java for | |
481 | a long time. | |
482 | ||
483 | - Added lzma_microlzma_encoder() and lzma_microlzma_decoder(). | |
484 | It is used by erofs-utils and may be used by others too. | |
485 | ||
486 | The MicroLZMA format is a raw LZMA stream (without end marker) | |
487 | whose first byte (always 0x00) has been replaced with | |
488 | bitwise-negation of the LZMA properties (lc/lp/pb). It was | |
489 | created for use in EROFS but may be used in other contexts | |
490 | as well where it is important to avoid wasting bytes for | |
491 | stream headers or footers. The format is also supported by | |
492 | XZ Embedded (the XZ Embedded version in Linux got MicroLZMA | |
493 | support in Linux 5.16). | |
494 | ||
495 | The MicroLZMA encoder API in liblzma can compress into a | |
496 | fixed-sized output buffer so that as much data is compressed | |
497 | as can be fit into the buffer while still creating a valid | |
498 | MicroLZMA stream. This is needed for EROFS. | |
499 | ||
500 | - Added lzma_lzip_decoder() to decompress the .lz (lzip) file | |
501 | format version 0 and the original unextended version 1 files. | |
502 | Also lzma_auto_decoder() supports .lz files. | |
503 | ||
504 | - lzma_filters_update() can now be used with the multi-threaded | |
505 | encoder (lzma_stream_encoder_mt()) to change the filter chain | |
506 | after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH. | |
507 | ||
508 | - In lzma_options_lzma, allow nice_len = 2 and 3 with the match | |
509 | finders that require at least 3 or 4. Now it is internally | |
510 | rounded up if needed. | |
511 | ||
512 | - CLMUL-based CRC64 on x86-64 and E2K with runtime processor | |
513 | detection. On 32-bit x86 it currently isn't available unless | |
514 | --disable-assembler is used which can make the non-CLMUL | |
515 | CRC64 slower; this might be fixed in the future. | |
516 | ||
517 | - Building with --disable-threads --enable-small | |
518 | is now thread-safe if the compiler supports | |
519 | __attribute__((__constructor__)). | |
520 | ||
521 | * xz: | |
522 | ||
523 | - Using -T0 (--threads=0) will now use multi-threaded encoder | |
524 | even on a single-core system. This is to ensure that output | |
525 | from the same xz binary is identical on both single-core and | |
526 | multi-core systems. | |
527 | ||
528 | - --threads=+1 or -T+1 is now a way to put xz into | |
529 | multi-threaded mode while using only one worker thread. | |
530 | The + is ignored if the number is not 1. | |
531 | ||
532 | - A default soft memory usage limit is now used for compression | |
533 | when -T0 is used and no explicit limit has been specified. | |
534 | This soft limit is used to restrict the number of threads | |
535 | but if the limit is exceeded with even one thread then xz | |
536 | will continue with one thread using the multi-threaded | |
537 | encoder and this limit is ignored. If the number of threads | |
538 | is specified manually then no default limit will be used; | |
539 | this affects only -T0. | |
540 | ||
541 | This change helps on systems that have very many cores and | |
542 | using all of them for xz makes no sense. Previously xz -T0 | |
543 | could run out of memory on such systems because it attempted | |
544 | to reserve memory for too many threads. | |
545 | ||
546 | This also helps with 32-bit builds which don't have a large | |
547 | amount of address space that would be required for many | |
548 | threads. The default soft limit for -T0 is at most 1400 MiB | |
549 | on all 32-bit platforms. | |
550 | ||
551 | - Previously a low value in --memlimit-compress wouldn't cause | |
552 | xz to switch from multi-threaded mode to single-threaded mode | |
553 | if the limit cannot otherwise be met; xz failed instead. Now | |
554 | xz can switch to single-threaded mode and then, if needed, | |
555 | scale down the LZMA2 dictionary size too just like it already | |
556 | did when it was started in single-threaded mode. | |
557 | ||
558 | - The option --no-adjust no longer prevents xz from scaling down | |
559 | the number of threads as that doesn't affect the compressed | |
560 | output (only performance). Now --no-adjust only prevents | |
561 | adjustments that affect compressed output, that is, with | |
562 | --no-adjust xz won't switch from multi-threaded mode to | |
563 | single-threaded mode and won't scale down the LZMA2 | |
564 | dictionary size. | |
565 | ||
566 | - Added a new option --memlimit-mt-decompress=LIMIT. This is | |
567 | used to limit the number of decompressor threads (possibly | |
568 | falling back to single-threaded mode) but it will never make | |
569 | xz refuse to decompress a file. This has a system-specific | |
570 | default value because without any limit xz could end up | |
571 | allocating memory for the whole compressed input file, the | |
572 | whole uncompressed output file, multiple thread-specific | |
573 | decompressor instances and so on. Basically xz could | |
574 | attempt to use an insane amount of memory even with fairly | |
575 | common files. The system-specific default value is currently | |
576 | the same as the one used for compression with -T0. | |
577 | ||
578 | The new option works together with the existing option | |
579 | --memlimit-decompress=LIMIT. The old option sets a hard limit | |
580 | that must not be exceeded (xz will refuse to decompress) | |
581 | while the new option only restricts the number of threads. | |
582 | If the limit set with --memlimit-mt-decompress is greater | |
583 | than the limit set with --memlimit-compress, then the latter | |
584 | value is used also for --memlimit-mt-decompress. | |
585 | ||
586 | - Added new information to the output of xz --info-memory and | |
587 | new fields to the output of xz --robot --info-memory. | |
588 | ||
589 | - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders | |
590 | now that liblzma handles it. | |
591 | ||
592 | - Don't mention endianness for ARM and ARM-Thumb filters in | |
593 | --long-help. The filters only work for little endian | |
594 | instruction encoding but modern ARM processors using | |
595 | big endian data access still use little endian | |
596 | instruction encoding. So the help text was misleading. | |
597 | In contrast, the PowerPC filter is only for big endian | |
598 | 32/64-bit PowerPC code. Little endian PowerPC would need | |
599 | a separate filter. | |
600 | ||
601 | - Added decompression support for the .lz (lzip) file format | |
602 | version 0 and the original unextended version 1. It is | |
603 | autodetected by default. See also the option --format on | |
604 | the xz man page. | |
605 | ||
606 | - Sandboxing enabled by default: | |
607 | * Capsicum (FreeBSD) | |
608 | * pledge(2) (OpenBSD) | |
609 | ||
610 | * Scripts now support the .lz format using xz. | |
611 | ||
612 | * A few new tests were added. | |
613 | ||
614 | * The liblzma-specific tests are now supported in CMake-based | |
615 | builds too ("make test"). | |
616 | ||
617 | ||
7e53c5bc LC |
618 | 5.3.5beta (2022-12-01) |
619 | ||
620 | * All fixes from 5.2.9. | |
621 | ||
622 | * liblzma: | |
623 | ||
624 | - Added new LZMA_FILTER_LZMA1EXT for raw encoder and decoder to | |
625 | handle raw LZMA1 streams that don't have end of payload marker | |
626 | (EOPM) alias end of stream (EOS) marker. It can be used in | |
627 | filter chains, for example, with the x86 BCJ filter. | |
628 | ||
629 | - Added lzma_str_to_filters(), lzma_str_from_filters(), and | |
630 | lzma_str_list_filters() to make it easier for applications | |
631 | to get custom compression options from a user and convert | |
632 | it to an array of lzma_filter structures. | |
633 | ||
634 | - Added lzma_filters_free(). | |
635 | ||
636 | - lzma_filters_update() can now be used with the multi-threaded | |
637 | encoder (lzma_stream_encoder_mt()) to change the filter chain | |
638 | after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH. | |
639 | ||
640 | - In lzma_options_lzma, allow nice_len = 2 and 3 with the match | |
641 | finders that require at least 3 or 4. Now it is internally | |
642 | rounded up if needed. | |
643 | ||
644 | - ARM64 filter was modified. It is still experimental. | |
645 | ||
646 | - Fixed LTO build with Clang if -fgnuc-version=10 or similar | |
647 | was used to make Clang look like GCC >= 10. Now it uses | |
648 | __has_attribute(__symver__) which should be reliable. | |
649 | ||
650 | * xz: | |
651 | ||
652 | - --threads=+1 or -T+1 is now a way to put xz into multi-threaded | |
653 | mode while using only one worker thread. | |
654 | ||
655 | - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders | |
656 | now that liblzma handles it. | |
657 | ||
658 | * Updated translations: Chinese (simplified), Korean, and Turkish. | |
659 | ||
660 | ||
5b999ba2 LC |
661 | 5.3.4alpha (2022-11-15) |
662 | ||
663 | * All fixes from 5.2.7 and 5.2.8. | |
664 | ||
665 | * liblzma: | |
666 | ||
667 | - Minor improvements to the threaded decoder. | |
668 | ||
669 | - Added CRC64 implementation that uses SSSE3, SSE4.1, and CLMUL | |
670 | instructions on 32/64-bit x86 and E2K. On 32-bit x86 it's | |
671 | not enabled unless --disable-assembler is used but then | |
672 | the non-CLMUL code might be slower. Processor support is | |
673 | detected at runtime so this is built by default on x86-64 | |
674 | and E2K. On these platforms, if compiler flags indicate | |
675 | unconditional CLMUL support (-msse4.1 -mpclmul) then the | |
676 | generic version is not built, making liblzma 8-9 KiB smaller | |
677 | compared to having both versions included. | |
678 | ||
679 | With extremely compressible files this can make decompression | |
680 | up to twice as fast but with typical files 5 % improvement | |
681 | is a more realistic expectation. | |
682 | ||
683 | The CLMUL version is slower than the generic version with | |
684 | tiny inputs (especially at 1-8 bytes per call, but up to | |
685 | 16 bytes). In normal use in xz this doesn't matter at all. | |
686 | ||
687 | - Added an experimental ARM64 filter. This is *not* the final | |
688 | version! Files created with this experimental version won't | |
689 | be supported in the future versions! The filter design is | |
690 | a compromise where improving one use case makes some other | |
691 | cases worse. | |
692 | ||
693 | - Added decompression support for the .lz (lzip) file format | |
694 | version 0 and the original unextended version 1. See the | |
695 | API docs of lzma_lzip_decoder() for details. Also | |
696 | lzma_auto_decoder() supports .lz files. | |
697 | ||
698 | - Building with --disable-threads --enable-small | |
699 | is now thread-safe if the compiler supports | |
700 | __attribute__((__constructor__)) | |
701 | ||
702 | * xz: | |
703 | ||
704 | - Added support for OpenBSD's pledge(2) as a sandboxing method. | |
705 | ||
706 | - Don't mention endianness for ARM and ARM-Thumb filters in | |
707 | --long-help. The filters only work for little endian | |
708 | instruction encoding but modern ARM processors using | |
709 | big endian data access still use little endian | |
710 | instruction encoding. So the help text was misleading. | |
711 | In contrast, the PowerPC filter is only for big endian | |
712 | 32/64-bit PowerPC code. Little endian PowerPC would need | |
713 | a separate filter. | |
714 | ||
715 | - Added --experimental-arm64. This will be renamed once the | |
716 | filter is finished. Files created with this experimental | |
717 | filter will not be supported in the future! | |
718 | ||
719 | - Added new fields to the output of xz --robot --info-memory. | |
720 | ||
721 | - Added decompression support for the .lz (lzip) file format | |
722 | version 0 and the original unextended version 1. It is | |
723 | autodetected by default. See also the option --format on | |
724 | the xz man page. | |
725 | ||
726 | * Scripts now support the .lz format using xz. | |
727 | ||
728 | * Build systems: | |
729 | ||
730 | - New #defines in config.h: HAVE_ENCODER_ARM64, | |
731 | HAVE_DECODER_ARM64, HAVE_LZIP_DECODER, HAVE_CPUID_H, | |
732 | HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR, HAVE_USABLE_CLMUL | |
733 | ||
734 | - New configure options: --disable-clmul-crc, | |
735 | --disable-microlzma, --disable-lzip-decoder, and | |
736 | 'pledge' is now an option in --enable-sandbox (but | |
737 | it's autodetected by default anyway). | |
738 | ||
739 | - INSTALL was updated to document the new configure options. | |
740 | ||
741 | - PACKAGERS now lists also --disable-microlzma and | |
742 | --disable-lzip-decoder as configure options that must | |
743 | not be used in builds for non-embedded use. | |
744 | ||
745 | * Tests: | |
746 | ||
747 | - Fix some of the tests so that they skip instead of fail if | |
748 | certain features have been disabled with configure options. | |
749 | It's still not perfect. | |
750 | ||
751 | - Other improvements to tests. | |
752 | ||
753 | * Updated translations: Croatian, Finnish, Hungarian, Polish, | |
754 | Romanian, Spanish, Swedish, and Ukrainian. | |
755 | ||
756 | ||
44fedddc LC |
757 | 5.3.3alpha (2022-08-22) |
758 | ||
759 | * All fixes from 5.2.6. | |
760 | ||
761 | * liblzma: | |
762 | ||
763 | - Fixed 32-bit build. | |
764 | ||
765 | - Added threaded .xz decompressor lzma_stream_decoder_mt(). | |
766 | It can use multiple threads with .xz files that have multiple | |
767 | Blocks with size information in Block Headers. The threaded | |
768 | encoder in xz has always created such files. | |
769 | ||
770 | Single-threaded encoder cannot store the size information in | |
771 | Block Headers even if one used LZMA_FULL_FLUSH to create | |
772 | multiple Blocks, so this threaded decoder cannot use multiple | |
773 | threads with such files. | |
774 | ||
775 | If there are multiple Streams (concatenated .xz files), one | |
776 | Stream will be decompressed completely before starting the | |
777 | next Stream. | |
778 | ||
779 | - A new decoder flag LZMA_FAIL_FAST was added. It makes the | |
780 | threaded decompressor report errors soon instead of first | |
781 | flushing all pending data before the error location. | |
782 | ||
783 | * xz: | |
784 | ||
785 | - Using -T0 (--threads=0) will now use multi-threaded encoder | |
786 | even on a single-core system. This is to ensure that output | |
787 | from the same xz binary is identical on both single-core and | |
788 | multi-core systems. | |
789 | ||
790 | - A default soft memory usage limit is now used for compression | |
791 | when -T0 is used and no explicit limit has been specified. | |
792 | This soft limit is used to restrict the number of threads | |
793 | but if the limit is exceeded with even one thread then xz | |
794 | will continue with one thread using the multi-threaded | |
795 | encoder and this limit is ignored. If the number of threads | |
796 | is specified manually then no default limit will be used; | |
797 | this affects only -T0. | |
798 | ||
799 | This change helps on systems that have very many cores and | |
800 | using all of them for xz makes no sense. Previously xz -T0 | |
801 | could run out of memory on such systems because it attempted | |
802 | to reserve memory for too many threads. | |
803 | ||
804 | This also helps with 32-bit builds which don't have a large | |
805 | amount of address space that would be required for many | |
806 | threads. The default limit is 1400 MiB on all 32-bit | |
807 | platforms with -T0. | |
808 | ||
809 | Now xz -T0 should just work. It might use too few threads | |
810 | in some cases but at least it shouldn't easily run out of | |
811 | memory. It's possible that this will be tweaked before 5.4.0. | |
812 | ||
813 | - Changes to --memlimit-compress and --no-adjust: | |
814 | ||
815 | In single-threaded mode, --memlimit-compress can make xz | |
816 | scale down the LZMA2 dictionary size to meet the memory usage | |
817 | limit. This obviously affects the compressed output. However, | |
818 | if xz was in threaded mode, --memlimit-compress could make xz | |
819 | reduce the number of threads but it wouldn't make xz switch | |
820 | from multi-threaded mode to single-threaded mode or scale | |
821 | down the LZMA2 dictionary size. This seemed illogical. | |
822 | ||
823 | Now --memlimit-compress can make xz switch to single-threaded | |
824 | mode if one thread in multi-threaded mode uses too much | |
825 | memory. If memory usage is still too high, then the LZMA2 | |
826 | dictionary size can be scaled down too. | |
827 | ||
828 | The option --no-adjust was also changed so that it no longer | |
829 | prevents xz from scaling down the number of threads as that | |
830 | doesn't affect compressed output (only performance). After | |
831 | this commit --no-adjust only prevents adjustments that affect | |
832 | compressed output, that is, with --no-adjust xz won't switch | |
833 | from multithreaded mode to single-threaded mode and won't | |
834 | scale down the LZMA2 dictionary size. | |
835 | ||
c3e94d37 | 836 | - Added a new option --memlimit-mt-decompress=LIMIT. This is |
44fedddc LC |
837 | used to limit the number of decompressor threads (possibly |
838 | falling back to single-threaded mode) but it will never make | |
839 | xz refuse to decompress a file. This has a system-specific | |
840 | default value because without any limit xz could end up | |
841 | allocating memory for the whole compressed input file, the | |
842 | whole uncompressed output file, multiple thread-specific | |
843 | decompressor instances and so on. Basically xz could | |
844 | attempt to use an insane amount of memory even with fairly | |
845 | common files. | |
846 | ||
847 | The new option works together with the existing option | |
848 | --memlimit-decompress=LIMIT. The old option sets a hard limit | |
849 | that must not be exceeded (xz will refuse to decompress) | |
850 | while the new option only restricts the number of threads. | |
851 | If the limit set with --memlimit-mt-decompress is greater | |
852 | than the limit set with --memlimit-compress, then the latter | |
853 | value is used also for --memlimit-mt-decompress. | |
854 | ||
855 | * Tests: | |
856 | ||
857 | - Added a few more tests. | |
858 | ||
859 | - Added tests/code_coverage.sh to create a code coverage report | |
860 | of the tests. | |
861 | ||
862 | * Build systems: | |
863 | ||
864 | - Automake's parallel test harness is now used to make tests | |
865 | finish faster. | |
866 | ||
867 | - Added the CMake files to the distribution tarball. These were | |
868 | supposed to be in 5.2.5 already. | |
869 | ||
870 | - Added liblzma tests to the CMake build. | |
871 | ||
872 | - Windows: Fix building of liblzma.dll with the included | |
873 | Visual Studio project files. | |
874 | ||
875 | ||
ea8c9486 LC |
876 | 5.3.2alpha (2021-10-28) |
877 | ||
878 | This release was made on short notice so that recent erofs-utils can | |
879 | be built with LZMA support without needing a snapshot from xz.git. | |
880 | Thus many pending things were not included, not even updated | |
881 | translations (which would need to be updated for the new --list | |
882 | strings anyway). | |
883 | ||
884 | * All fixes from 5.2.5. | |
885 | ||
886 | * xz: | |
887 | ||
888 | - When copying metadata from the source file to the destination | |
889 | file, don't try to set the group (GID) if it is already set | |
890 | correctly. This avoids a failure on OpenBSD (and possibly on | |
891 | a few other OSes) where files may get created so that their | |
892 | group doesn't belong to the user, and fchown(2) can fail even | |
893 | if it needs to do nothing. | |
894 | ||
895 | - The --keep option now accepts symlinks, hardlinks, and | |
896 | setuid, setgid, and sticky files. Previously this required | |
897 | using --force. | |
898 | ||
899 | - Split the long strings used in --list and --info-memory modes | |
900 | to make them much easier for translators. | |
901 | ||
902 | - If built with sandbox support and enabling the sandbox fails, | |
903 | xz will now immediately exit with exit status of 1. Previously | |
904 | it would only display a warning if -vv was used. | |
905 | ||
906 | - Cap --memlimit-compress to 2000 MiB on MIPS32 because on | |
907 | MIPS32 userspace processes are limited to 2 GiB of address | |
908 | space. | |
909 | ||
910 | * liblzma: | |
911 | ||
912 | - Added lzma_microlzma_encoder() and lzma_microlzma_decoder(). | |
913 | The API is in lzma/container.h. | |
914 | ||
915 | The MicroLZMA format is a raw LZMA stream (without end marker) | |
916 | whose first byte (always 0x00) has been replaced with | |
917 | bitwise-negation of the LZMA properties (lc/lp/pb). It was | |
918 | created for use in EROFS but may be used in other contexts | |
919 | as well where it is important to avoid wasting bytes for | |
920 | stream headers or footers. The format is also supported by | |
921 | XZ Embedded. | |
922 | ||
923 | The MicroLZMA encoder API in liblzma can compress into a | |
924 | fixed-sized output buffer so that as much data is compressed | |
925 | as can be fit into the buffer while still creating a valid | |
926 | MicroLZMA stream. This is needed for EROFS. | |
927 | ||
928 | - Added fuzzing support. | |
929 | ||
930 | - Support Intel Control-flow Enforcement Technology (CET) in | |
931 | 32-bit x86 assembly files. | |
932 | ||
933 | - Visual Studio: Use non-standard _MSVC_LANG to detect C++ | |
934 | standard version in the lzma.h API header. It's used to | |
935 | detect when "noexcept" can be used. | |
936 | ||
937 | * Scripts: | |
938 | ||
939 | - Fix exit status of xzdiff/xzcmp. Exit status could be 2 when | |
940 | the correct value is 1. | |
941 | ||
942 | - Fix exit status of xzgrep. | |
943 | ||
944 | - Detect corrupt .bz2 files in xzgrep. | |
945 | ||
946 | - Add zstd support to xzgrep and xzdiff/xzcmp. | |
947 | ||
948 | - Fix less(1) version detection in xzless. It failed if the | |
949 | version number from "less -V" contained a dot. | |
950 | ||
951 | * Fix typos and technical issues in man pages. | |
952 | ||
953 | * Build systems: | |
954 | ||
955 | - Windows: Fix building of resource files when config.h isn't | |
956 | used. CMake + Visual Studio can now build liblzma.dll. | |
957 | ||
958 | - Various fixes to the CMake support. It might still need a few | |
959 | more fixes even for liblzma-only builds. | |
960 | ||
961 | ||
114cab97 LC |
962 | 5.3.1alpha (2018-04-29) |
963 | ||
964 | * All fixes from 5.2.4. | |
965 | ||
966 | * Add lzma_file_info_decoder() into liblzma and use it in xz to | |
967 | implement the --list feature. | |
968 | ||
969 | * Capsicum sandbox support is enabled by default where available | |
970 | (FreeBSD >= 10). | |
971 | ||
972 | ||
77050b78 JT |
973 | 5.2.12 (2023-05-04) |
974 | ||
975 | * Fixed a build system bug that prevented building liblzma as a | |
976 | shared library when configured with --disable-threads. This bug | |
977 | affected releases 5.2.6 to 5.2.11 and 5.4.0 to 5.4.2. | |
978 | ||
979 | * Include <intrin.h> for Windows intrinsic functions where they are | |
980 | needed. This fixed a bug that prevented building liblzma using | |
981 | clang-cl on Windows. | |
982 | ||
983 | * Minor update to the Croatian translation. The small change | |
984 | applies to a string in both 5.2 and 5.4 branches. | |
985 | ||
986 | ||
af4925e6 JT |
987 | 5.2.11 (2023-03-18) |
988 | ||
989 | * Removed all possible cases of null pointer + 0. It is undefined | |
990 | behavior in C99 and C17. This was detected by a sanitizer and had | |
991 | not caused any known issues. | |
992 | ||
993 | * Build systems: | |
994 | ||
995 | - Added a workaround for building with GCC on MicroBlaze Linux. | |
996 | GCC 12 on MicroBlaze doesn't support the __symver__ attribute | |
997 | even though __has_attribute(__symver__) returns true. The | |
998 | build is now done without the extra RHEL/CentOS 7 symbols | |
999 | that were added in XZ Utils 5.2.7. The workaround only | |
1000 | applies to the Autotools build (not CMake). | |
1001 | ||
1002 | - CMake: Ensure that the C compiler language is set to C99 or | |
1003 | a newer standard. | |
1004 | ||
1005 | - CMake changes from XZ Utils 5.4.1: | |
1006 | ||
1007 | * Added a workaround for a build failure with | |
1008 | windres from GNU binutils. | |
1009 | ||
1010 | * Included the Windows resource files in the xz | |
1011 | and xzdec build rules. | |
1012 | ||
1013 | ||
0d2a2e0a LC |
1014 | 5.2.10 (2022-12-13) |
1015 | ||
1016 | * xz: Don't modify argv[] when parsing the --memlimit* and | |
1017 | --block-list command line options. This fixes confusing | |
1018 | arguments in process listing (like "ps auxf"). | |
1019 | ||
1020 | * GNU/Linux only: Use __has_attribute(__symver__) to detect if | |
1021 | that attribute is supported. This fixes build on Mandriva where | |
1022 | Clang is patched to define __GNUC__ to 11 by default (instead | |
1023 | of 4 as used by Clang upstream). | |
1024 | ||
1025 | ||
3be6942e LC |
1026 | 5.2.9 (2022-11-30) |
1027 | ||
1028 | * liblzma: | |
1029 | ||
1030 | - Fixed an infinite loop in LZMA encoder initialization | |
1031 | if dict_size >= 2 GiB. (The encoder only supports up | |
1032 | to 1536 MiB.) | |
1033 | ||
1034 | - Fixed two cases of invalid free() that can happen if | |
1035 | a tiny allocation fails in encoder re-initialization | |
1036 | or in lzma_filters_update(). These bugs had some | |
1037 | similarities with the bug fixed in 5.2.7. | |
1038 | ||
1039 | - Fixed lzma_block_encoder() not allowing the use of | |
1040 | LZMA_SYNC_FLUSH with lzma_code() even though it was | |
1041 | documented to be supported. The sync-flush code in | |
1042 | the Block encoder was already used internally via | |
1043 | lzma_stream_encoder(), so this was just a missing flag | |
1044 | in the lzma_block_encoder() API function. | |
1045 | ||
1046 | - GNU/Linux only: Don't put symbol versions into static | |
1047 | liblzma as it breaks things in some cases (and even if | |
1048 | it didn't break anything, symbol versions in static | |
1049 | libraries are useless anyway). The downside of the fix | |
1050 | is that if the configure options --with-pic or --without-pic | |
1051 | are used then it's not possible to build both shared and | |
1052 | static liblzma at the same time on GNU/Linux anymore; | |
1053 | with those options --disable-static or --disable-shared | |
1054 | must be used too. | |
1055 | ||
1056 | * New email address for bug reports is <xz@tukaani.org> which | |
1057 | forwards messages to Lasse Collin and Jia Tan. | |
1058 | ||
1059 | ||
ce8db9e3 LC |
1060 | 5.2.8 (2022-11-13) |
1061 | ||
1062 | * xz: | |
1063 | ||
1064 | - If xz cannot remove an input file when it should, this | |
1065 | is now treated as a warning (exit status 2) instead of | |
1066 | an error (exit status 1). This matches GNU gzip and it | |
1067 | is more logical as at that point the output file has | |
1068 | already been successfully closed. | |
1069 | ||
1070 | - Fix handling of .xz files with an unsupported check type. | |
1071 | Previously such printed a warning message but then xz | |
1072 | behaved as if an error had occurred (didn't decompress, | |
1073 | exit status 1). Now a warning is printed, decompression | |
1074 | is done anyway, and exit status is 2. This used to work | |
1075 | slightly before 5.0.0. In practice this bug matters only | |
1076 | if xz has been built with some check types disabled. As | |
1077 | instructed in PACKAGERS, such builds should be done in | |
1078 | special situations only. | |
1079 | ||
1080 | - Fix "xz -dc --single-stream tests/files/good-0-empty.xz" | |
1081 | which failed with "Internal error (bug)". That is, | |
1082 | --single-stream was broken if the first .xz stream in | |
1083 | the input file didn't contain any uncompressed data. | |
1084 | ||
1085 | - Fix displaying file sizes in the progress indicator when | |
1086 | working in passthru mode and there are multiple input files. | |
1087 | Just like "gzip -cdf", "xz -cdf" works like "cat" when the | |
1088 | input file isn't a supported compressed file format. In | |
1089 | this case the file size counters weren't reset between | |
1090 | files so with multiple input files the progress indicator | |
1091 | displayed an incorrect (too large) value. | |
1092 | ||
1093 | * liblzma: | |
1094 | ||
1095 | - API docs in lzma/container.h: | |
1096 | * Update the list of decoder flags in the decoder | |
1097 | function docs. | |
1098 | * Explain LZMA_CONCATENATED behavior with .lzma files | |
1099 | in lzma_auto_decoder() docs. | |
1100 | ||
1101 | - OpenBSD: Use HW_NCPUONLINE to detect the number of | |
1102 | available hardware threads in lzma_physmem(). | |
1103 | ||
1104 | - Fix use of wrong macro to detect x86 SSE2 support. | |
1105 | __SSE2_MATH__ was used with GCC/Clang but the correct | |
1106 | one is __SSE2__. The first one means that SSE2 is used | |
1107 | for floating point math which is irrelevant here. | |
1108 | The affected SSE2 code isn't used on x86-64 so this affects | |
1109 | only 32-bit x86 builds that use -msse2 without -mfpmath=sse | |
1110 | (there is no runtime detection for SSE2). It improves LZMA | |
1111 | compression speed (not decompression). | |
1112 | ||
1113 | - Fix the build with Intel C compiler 2021 (ICC, not ICX) | |
1114 | on Linux. It defines __GNUC__ to 10 but doesn't support | |
1115 | the __symver__ attribute introduced in GCC 10. | |
1116 | ||
1117 | * Scripts: Ignore warnings from xz by using --quiet --no-warn. | |
1118 | This is needed if the input .xz files use an unsupported | |
1119 | check type. | |
1120 | ||
1121 | * Translations: | |
1122 | ||
1123 | - Updated Croatian and Turkish translations. | |
1124 | ||
1125 | - One new translations wasn't included because it needed | |
1126 | technical fixes. It will be in upcoming 5.4.0. No new | |
1127 | translations will be added to the 5.2.x branch anymore. | |
1128 | ||
1129 | - Renamed the French man page translation file from | |
1130 | fr_FR.po to fr.po and thus also its install directory | |
1131 | (like /usr/share/man/fr_FR -> .../fr). | |
1132 | ||
1133 | - Man page translations for upcoming 5.4.0 are now handled | |
1134 | in the Translation Project. | |
1135 | ||
1136 | * Update doc/faq.txt a little so it's less out-of-date. | |
1137 | ||
1138 | ||
bd334ae5 LC |
1139 | 5.2.7 (2022-09-30) |
1140 | ||
1141 | * liblzma: | |
1142 | ||
1143 | - Made lzma_filters_copy() to never modify the destination | |
1144 | array if an error occurs. lzma_stream_encoder() and | |
1145 | lzma_stream_encoder_mt() already assumed this. Before this | |
1146 | change, if a tiny memory allocation in lzma_filters_copy() | |
1147 | failed it would lead to a crash (invalid free() or invalid | |
1148 | memory reads) in the cleanup paths of these two encoder | |
1149 | initialization functions. | |
1150 | ||
1151 | - Added missing integer overflow check to lzma_index_append(). | |
1152 | This affects xz --list and other applications that decode | |
1153 | the Index field from .xz files using lzma_index_decoder(). | |
1154 | Normal decompression of .xz files doesn't call this code | |
1155 | and thus most applications using liblzma aren't affected | |
1156 | by this bug. | |
1157 | ||
1158 | - Single-threaded .xz decoder (lzma_stream_decoder()): If | |
1159 | lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible | |
1160 | to use lzma_memlimit_set() to increase the limit and continue | |
1161 | decoding. This was supposed to work from the beginning | |
1162 | but there was a bug. With other decoders (.lzma or | |
1163 | threaded .xz decoder) this already worked correctly. | |
1164 | ||
1165 | - Fixed accumulation of integrity check type statistics in | |
1166 | lzma_index_cat(). This bug made lzma_index_checks() return | |
1167 | only the type of the integrity check of the last Stream | |
1168 | when multiple lzma_indexes were concatenated. Most | |
1169 | applications don't use these APIs but in xz it made | |
1170 | xz --list not list all check types from concatenated .xz | |
1171 | files. In xz --list --verbose only the per-file "Check:" | |
1172 | lines were affected and in xz --robot --list only the "file" | |
1173 | line was affected. | |
1174 | ||
1175 | - Added ABI compatibility with executables that were linked | |
1176 | against liblzma in RHEL/CentOS 7 or other liblzma builds | |
1177 | that had copied the problematic patch from RHEL/CentOS 7 | |
1178 | (xz-5.2.2-compat-libs.patch). For the details, see the | |
1179 | comment at the top of src/liblzma/validate_map.sh. | |
1180 | ||
1181 | WARNING: This uses __symver__ attribute with GCC >= 10. | |
1182 | In other cases the traditional __asm__(".symver ...") | |
1183 | is used. Using link-time optimization (LTO, -flto) with | |
1184 | GCC versions older than 10 can silently result in | |
1185 | broken liblzma.so.5 (incorrect symbol versions)! If you | |
1186 | want to use -flto with GCC, you must use GCC >= 10. | |
1187 | LTO with Clang seems to work even with the traditional | |
1188 | __asm__(".symver ...") method. | |
1189 | ||
1190 | * xzgrep: Fixed compatibility with old shells that break if | |
1191 | comments inside command substitutions have apostrophes ('). | |
1192 | This problem was introduced in 5.2.6. | |
1193 | ||
1194 | * Build systems: | |
1195 | ||
1196 | - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX | |
1197 | ||
1198 | - Windows: Fixed liblzma.dll build with Visual Studio project | |
1199 | files. It broke in 5.2.6 due to a change that was made to | |
1200 | improve CMake support. | |
1201 | ||
1202 | - Windows: Building liblzma with UNICODE defined should now | |
1203 | work. | |
1204 | ||
1205 | - CMake files are now actually included in the release tarball. | |
1206 | They should have been in 5.2.5 already. | |
1207 | ||
1208 | - Minor CMake fixes and improvements. | |
1209 | ||
1210 | * Added a new translation: Turkish | |
1211 | ||
1212 | ||
6dcf606e LC |
1213 | 5.2.6 (2022-08-12) |
1214 | ||
1215 | * xz: | |
1216 | ||
1217 | - The --keep option now accepts symlinks, hardlinks, and | |
1218 | setuid, setgid, and sticky files. Previously this required | |
1219 | using --force. | |
1220 | ||
1221 | - When copying metadata from the source file to the destination | |
1222 | file, don't try to set the group (GID) if it is already set | |
1223 | correctly. This avoids a failure on OpenBSD (and possibly on | |
1224 | a few other OSes) where files may get created so that their | |
1225 | group doesn't belong to the user, and fchown(2) can fail even | |
1226 | if it needs to do nothing. | |
1227 | ||
1228 | - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on | |
1229 | MIPS32 because on MIPS32 userspace processes are limited | |
1230 | to 2 GiB of address space. | |
1231 | ||
1232 | * liblzma: | |
1233 | ||
1234 | - Fixed a missing error-check in the threaded encoder. If a | |
1235 | small memory allocation fails, a .xz file with an invalid | |
1236 | Index field would be created. Decompressing such a file would | |
1237 | produce the correct output but result in an error at the end. | |
1238 | Thus this is a "mild" data corruption bug. Note that while | |
1239 | a failed memory allocation can trigger the bug, it cannot | |
1240 | cause invalid memory access. | |
1241 | ||
1242 | - The decoder for .lzma files now supports files that have | |
1243 | uncompressed size stored in the header and still use the | |
1244 | end of payload marker (end of stream marker) at the end | |
1245 | of the LZMA stream. Such files are rare but, according to | |
1246 | the documentation in LZMA SDK, they are valid. | |
1247 | doc/lzma-file-format.txt was updated too. | |
1248 | ||
1249 | - Improved 32-bit x86 assembly files: | |
1250 | * Support Intel Control-flow Enforcement Technology (CET) | |
1251 | * Use non-executable stack on FreeBSD. | |
1252 | ||
1253 | - Visual Studio: Use non-standard _MSVC_LANG to detect C++ | |
1254 | standard version in the lzma.h API header. It's used to | |
1255 | detect when "noexcept" can be used. | |
1256 | ||
1257 | * xzgrep: | |
1258 | ||
1259 | - Fixed arbitrary command injection via a malicious filename | |
1260 | (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for | |
1261 | this was released to the public on 2022-04-07. A slight | |
1262 | robustness improvement has been made since then and, if | |
1263 | using GNU or *BSD grep, a new faster method is now used | |
1264 | that doesn't use the old sed-based construct at all. This | |
1265 | also fixes bad output with GNU grep >= 3.5 (2020-09-27) | |
1266 | when xzgrepping binary files. | |
1267 | ||
1268 | This vulnerability was discovered by: | |
1269 | cleemy desu wayo working with Trend Micro Zero Day Initiative | |
1270 | ||
1271 | - Fixed detection of corrupt .bz2 files. | |
1272 | ||
1273 | - Improved error handling to fix exit status in some situations | |
1274 | and to fix handling of signals: in some situations a signal | |
1275 | didn't make xzgrep exit when it clearly should have. It's | |
1276 | possible that the signal handling still isn't quite perfect | |
1277 | but hopefully it's good enough. | |
1278 | ||
1279 | - Documented exit statuses on the man page. | |
1280 | ||
1281 | - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead | |
1282 | of the deprecated egrep and fgrep commands. | |
1283 | ||
1284 | - Fixed parsing of the options -E, -F, -G, -P, and -X. The | |
42df7c7a | 1285 | problem occurred when multiple options were specified in |
6dcf606e LC |
1286 | a single argument, for example, |
1287 | ||
1288 | echo foo | xzgrep -Fe foo | |
1289 | ||
1290 | treated foo as a filename because -Fe wasn't correctly | |
1291 | split into -F -e. | |
1292 | ||
1293 | - Added zstd support. | |
1294 | ||
1295 | * xzdiff/xzcmp: | |
1296 | ||
1297 | - Fixed wrong exit status. Exit status could be 2 when the | |
1298 | correct value is 1. | |
1299 | ||
1300 | - Documented on the man page that exit status of 2 is used | |
1301 | for decompression errors. | |
1302 | ||
1303 | - Added zstd support. | |
1304 | ||
1305 | * xzless: | |
1306 | ||
1307 | - Fix less(1) version detection. It failed if the version number | |
1308 | from "less -V" contained a dot. | |
1309 | ||
1310 | * Translations: | |
1311 | ||
1312 | - Added new translations: Catalan, Croatian, Esperanto, | |
1313 | Korean, Portuguese, Romanian, Serbian, Spanish, Swedish, | |
1314 | and Ukrainian | |
1315 | ||
1316 | - Updated the Brazilian Portuguese translation. | |
1317 | ||
1318 | - Added French man page translation. This and the existing | |
1319 | German translation aren't complete anymore because the | |
1320 | English man pages got a few updates and the translators | |
1321 | weren't reached so that they could update their work. | |
1322 | ||
1323 | * Build systems: | |
1324 | ||
1325 | - Windows: Fix building of resource files when config.h isn't | |
1326 | used. CMake + Visual Studio can now build liblzma.dll. | |
1327 | ||
1328 | - Various fixes to the CMake support. Building static or shared | |
1329 | liblzma should work fine in most cases. In contrast, building | |
1330 | the command line tools with CMake is still clearly incomplete | |
1331 | and experimental and should be used for testing only. | |
1332 | ||
1333 | ||
869b9d1b LC |
1334 | 5.2.5 (2020-03-17) |
1335 | ||
1336 | * liblzma: | |
1337 | ||
1338 | - Fixed several C99/C11 conformance bugs. Now the code is clean | |
1339 | under gcc/clang -fsanitize=undefined. Some of these changes | |
1340 | might have a negative effect on performance with old GCC | |
1341 | versions or compilers other than GCC and Clang. The configure | |
1342 | option --enable-unsafe-type-punning can be used to (mostly) | |
1343 | restore the old behavior but it shouldn't normally be used. | |
1344 | ||
1345 | - Improved API documentation of lzma_properties_decode(). | |
1346 | ||
1347 | - Added a very minor encoder speed optimization. | |
1348 | ||
1349 | * xz: | |
1350 | ||
1351 | - Fixed a crash in "xz -dcfv not_an_xz_file". All four options | |
1352 | were required to trigger it. The crash occurred in the | |
1353 | progress indicator code when xz was in passthru mode where | |
1354 | xz works like "cat". | |
1355 | ||
1356 | - Fixed an integer overflow with 32-bit off_t. It could happen | |
1357 | when decompressing a file that has a long run of zero bytes | |
1358 | which xz would try to write as a sparse file. Since the build | |
1359 | system enables large file support by default, off_t is | |
1360 | normally 64-bit even on 32-bit systems. | |
1361 | ||
1362 | - Fixes for --flush-timeout: | |
1363 | * Fix semi-busy-waiting. | |
1364 | * Avoid unneeded flushes when no new input has arrived | |
1365 | since the previous flush was completed. | |
1366 | ||
1367 | - Added a special case for 32-bit xz: If --memlimit-compress is | |
1368 | used to specify a limit that exceeds 4020 MiB, the limit will | |
1369 | be set to 4020 MiB. The values "0" and "max" aren't affected | |
1370 | by this and neither is decompression. This hack can be | |
1371 | helpful when a 32-bit xz has access to 4 GiB address space | |
1372 | but the specified memlimit exceeds 4 GiB. This can happen | |
1373 | e.g. with some scripts. | |
1374 | ||
1375 | - Capsicum sandbox is now enabled by default where available | |
1376 | (FreeBSD >= 10). The sandbox debug messages (xz -vv) were | |
1377 | removed since they seemed to be more annoying than useful. | |
1378 | ||
1379 | - DOS build now requires DJGPP 2.05 instead of 2.04beta. | |
1380 | A workaround for a locale problem with DJGPP 2.05 was added. | |
1381 | ||
1382 | * xzgrep and other scripts: | |
1383 | ||
1384 | - Added a configure option --enable-path-for-scripts=PREFIX. | |
1385 | It is disabled by default except on Solaris where the default | |
1386 | is /usr/xpg4/bin. See INSTALL for details. | |
1387 | ||
1388 | - Added a workaround for a POSIX shell detection problem on | |
1389 | Solaris. | |
1390 | ||
1391 | * Build systems: | |
1392 | ||
1393 | - Added preliminary build instructions for z/OS. See INSTALL | |
1394 | section 1.2.9. | |
1395 | ||
1396 | - Experimental CMake support was added. It should work to build | |
1397 | static liblzma on a few operating systems. It may or may not | |
1398 | work to build shared liblzma. On some platforms it can build | |
1399 | xz and xzdec too but those are only for testing. See the | |
1400 | comment in the beginning of CMakeLists.txt for details. | |
1401 | ||
1402 | - Visual Studio project files were updated. | |
1403 | WindowsTargetPlatformVersion was removed from VS2017 files | |
1404 | and set to "10.0" in the added VS2019 files. In the future | |
1405 | the VS project files will be removed when CMake support is | |
1406 | good enough. | |
1407 | ||
1408 | - New #defines in config.h: HAVE___BUILTIN_ASSUME_ALIGNED, | |
1409 | HAVE___BUILTIN_BSWAPXX, and TUKLIB_USE_UNSAFE_TYPE_PUNNING. | |
1410 | ||
1411 | - autogen.sh has a new optional dependency on po4a and a new | |
1412 | option --no-po4a to skip that step. This matters only if one | |
1413 | wants to remake the build files. po4a is used to update the | |
1414 | translated man pages but as long as the man pages haven't | |
1415 | been modified, there's nothing to update and one can use | |
1416 | --no-po4a to avoid the dependency on po4a. | |
1417 | ||
1418 | * Translations: | |
1419 | ||
1420 | - XZ Utils translations are now handled by the Translation | |
1421 | Project: https://translationproject.org/domain/xz.html | |
1422 | ||
1423 | - All man pages are now included in German too. | |
1424 | ||
1425 | - New xz translations: Brazilian Portuguese, Finnish, | |
1426 | Hungarian, Chinese (simplified), Chinese (traditional), | |
1427 | and Danish (partial translation) | |
1428 | ||
1429 | - Updated xz translations: French, German, Italian, and Polish | |
1430 | ||
1431 | - Unfortunately a few new xz translations weren't included due | |
1432 | to technical problems like too long lines in --help output or | |
1433 | misaligned column headings in tables. In the future, many of | |
1434 | these strings will be split and e.g. the table column | |
1435 | alignment will be handled in software. This should make the | |
1436 | strings easier to translate. | |
1437 | ||
1438 | ||
b8139e11 LC |
1439 | 5.2.4 (2018-04-29) |
1440 | ||
1441 | * liblzma: | |
1442 | ||
1443 | - Allow 0 as memory usage limit instead of returning | |
1444 | LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified, | |
1445 | which effectively is the same as 0. | |
1446 | ||
1447 | - Use "noexcept" keyword instead of "throw()" in the public | |
1448 | headers when a C++11 (or newer standard) compiler is used. | |
1449 | ||
1450 | - Added a portability fix for recent Intel C Compilers. | |
1451 | ||
1452 | - Microsoft Visual Studio build files have been moved under | |
1453 | windows/vs2013 and windows/vs2017. | |
1454 | ||
1455 | * xz: | |
1456 | ||
1457 | - Fix "xz --list --robot missing_or_bad_file.xz" which would | |
2fb0ddaa | 1458 | try to print an uninitialized string and thus produce garbage |
b8139e11 LC |
1459 | output. Since the exit status is non-zero, most uses of such |
1460 | a command won't try to interpret the garbage output. | |
1461 | ||
1462 | - "xz --list foo.xz" could print "Internal error (bug)" in a | |
1463 | corner case where a specific memory usage limit had been set. | |
1464 | ||
1465 | ||
78ae13bc LC |
1466 | 5.2.3 (2016-12-30) |
1467 | ||
1468 | * xz: | |
1469 | ||
1470 | - Always close a file before trying to delete it to avoid | |
1471 | problems on some operating system and file system combinations. | |
1472 | ||
1473 | - Fixed copying of file timestamps on Windows. | |
1474 | ||
1475 | - Added experimental (disabled by default) sandbox support using | |
1476 | Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL. | |
1477 | ||
1478 | * C99/C11 conformance fixes to liblzma. The issues affected at least | |
1479 | some builds using link-time optimizations. | |
1480 | ||
1481 | * Fixed bugs in the rarely-used function lzma_index_dup(). | |
1482 | ||
1483 | * Use of external SHA-256 code is now disabled by default. | |
1484 | It can still be enabled by passing --enable-external-sha256 | |
1485 | to configure. The reasons to disable it by default (see INSTALL | |
1486 | for more details): | |
1487 | ||
1488 | - Some OS-specific SHA-256 implementations conflict with | |
1489 | OpenSSL and cause problems in programs that link against both | |
1490 | liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0 | |
1491 | are affected. | |
1492 | ||
1493 | - The internal SHA-256 is faster than the SHA-256 code in | |
1494 | some operating systems. | |
1495 | ||
1496 | * Changed CPU core count detection to use sched_getaffinity() on | |
1497 | GNU/Linux and GNU/kFreeBSD. | |
1498 | ||
1499 | * Fixes to the build-system and xz to make xz buildable even when | |
1500 | encoders, decoders, or threading have been disabled from libilzma | |
1501 | using configure options. These fixes added two new #defines to | |
1502 | config.h: HAVE_ENCODERS and HAVE_DECODERS. | |
1503 | ||
1504 | ||
7f058039 LC |
1505 | 5.2.2 (2015-09-29) |
1506 | ||
1507 | * Fixed bugs in QNX-specific code. | |
1508 | ||
1509 | * Omitted the use of pipe2() even if it is available to avoid | |
1510 | portability issues with some old Linux and glibc combinations. | |
1511 | ||
1512 | * Updated German translation. | |
1513 | ||
1514 | * Added project files to build static and shared liblzma (not the | |
1515 | whole XZ Utils) with Visual Studio 2013 update 2 or later. | |
1516 | ||
1517 | * Documented that threaded decompression hasn't been implemented | |
1518 | yet. A 5.2.0 NEWS entry describing multi-threading support had | |
1519 | incorrectly said "decompression" when it should have said | |
1520 | "compression". | |
1521 | ||
1522 | ||
29e39c79 LC |
1523 | 5.2.1 (2015-02-26) |
1524 | ||
1525 | * Fixed a compression-ratio regression in fast mode of LZMA1 and | |
1526 | LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases. | |
1527 | ||
1528 | * Fixed a portability problem in xz that affected at least OpenBSD. | |
1529 | ||
1530 | * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs | |
1531 | from most other mktemp implementations. | |
1532 | ||
1533 | * Changed CPU core count detection to use cpuset_getaffinity() on | |
1534 | FreeBSD. | |
1535 | ||
1536 | ||
f168a6fd LC |
1537 | 5.2.0 (2014-12-21) |
1538 | ||
1539 | Since 5.1.4beta: | |
1540 | ||
1541 | * All fixes from 5.0.8 | |
1542 | ||
1543 | * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset | |
1544 | was used. | |
1545 | ||
1546 | * xzdiff: If mktemp isn't installed, mkdir will be used as | |
1547 | a fallback to create a temporary directory. Installing mktemp | |
1548 | is still recommended. | |
1549 | ||
1550 | * Updated French, German, Italian, Polish, and Vietnamese | |
1551 | translations. | |
1552 | ||
1553 | Summary of fixes and new features added in the 5.1.x development | |
1554 | releases: | |
1555 | ||
1556 | * liblzma: | |
1557 | ||
1558 | - Added support for multi-threaded compression. See the | |
1559 | lzma_mt structure, lzma_stream_encoder_mt(), and | |
1560 | lzma_stream_encoder_mt_memusage() in <lzma/container.h>, | |
1561 | lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads() | |
1562 | in <lzma/hardware.h> for details. | |
1563 | ||
1564 | - Made the uses of lzma_allocator const correct. | |
1565 | ||
1566 | - Added lzma_block_uncomp_encode() to create uncompressed | |
1567 | .xz Blocks using LZMA2 uncompressed chunks. | |
1568 | ||
1569 | - Added support for LZMA_IGNORE_CHECK. | |
1570 | ||
1571 | - A few speed optimizations were made. | |
1572 | ||
1573 | - Added support for symbol versioning. It is enabled by default | |
1574 | on GNU/Linux, other GNU-based systems, and FreeBSD. | |
1575 | ||
1576 | - liblzma (not the whole XZ Utils) should now be buildable | |
1577 | with MSVC 2013 update 2 or later using windows/config.h. | |
1578 | ||
1579 | * xz: | |
1580 | ||
1581 | - Fixed a race condition in the signal handling. It was | |
1582 | possible that e.g. the first SIGINT didn't make xz exit | |
1583 | if reading or writing blocked and one had bad luck. The fix | |
1584 | is non-trivial, so as of writing it is unknown if it will be | |
1585 | backported to the v5.0 branch. | |
1586 | ||
17b29d4f | 1587 | - Multi-threaded compression can be enabled with the |
f168a6fd | 1588 | --threads (-T) option. |
17b29d4f | 1589 | [Fixed: This originally said "decompression".] |
f168a6fd LC |
1590 | |
1591 | - New command line options in xz: --single-stream, | |
1592 | --block-size=SIZE, --block-list=SIZES, | |
1593 | --flush-timeout=TIMEOUT, and --ignore-check. | |
1594 | ||
1595 | - xz -lvv now shows the minimum xz version that is required to | |
1596 | decompress the file. Currently it is 5.0.0 for all supported | |
1597 | .xz files except files with empty LZMA2 streams require 5.0.2. | |
1598 | ||
1599 | * xzdiff and xzgrep now support .lzo files if lzop is installed. | |
1600 | The .tzo suffix is also recognized as a shorthand for .tar.lzo. | |
1601 | ||
1602 | ||
e9e097e2 LC |
1603 | 5.1.4beta (2014-09-14) |
1604 | ||
1605 | * All fixes from 5.0.6 | |
1606 | ||
1607 | * liblzma: Fixed the use of presets in threaded encoder | |
1608 | initialization. | |
1609 | ||
1610 | * xz --block-list and --block-size can now be used together | |
1611 | in single-threaded mode. Previously the combination only | |
1612 | worked in multi-threaded mode. | |
1613 | ||
1614 | * Added support for LZMA_IGNORE_CHECK to liblzma and made it | |
1615 | available in xz as --ignore-check. | |
1616 | ||
1617 | * liblzma speed optimizations: | |
1618 | ||
1619 | - Initialization of a new LZMA1 or LZMA2 encoder has been | |
1620 | optimized. (The speed of reinitializing an already-allocated | |
1621 | encoder isn't affected.) This helps when compressing many | |
1622 | small buffers with lzma_stream_buffer_encode() and other | |
1623 | similar situations where an already-allocated encoder state | |
1624 | isn't reused. This speed-up is visible in xz too if one | |
1625 | compresses many small files one at a time instead running xz | |
1626 | once and giving all files as command-line arguments. | |
1627 | ||
1628 | - Buffer comparisons are now much faster when unaligned access | |
1629 | is allowed (configured with --enable-unaligned-access). This | |
1630 | speeds up encoding significantly. There is arch-specific code | |
1631 | for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best | |
1632 | results and there's no run-time CPU detection for now). | |
1633 | For other archs there is only generic code which probably | |
1634 | isn't as optimal as arch-specific solutions could be. | |
1635 | ||
1636 | - A few speed optimizations were made to the SHA-256 code. | |
1637 | (Note that the builtin SHA-256 code isn't used on all | |
1638 | operating systems.) | |
1639 | ||
1640 | * liblzma can now be built with MSVC 2013 update 2 or later | |
1641 | using windows/config.h. | |
1642 | ||
1643 | * Vietnamese translation was added. | |
1644 | ||
1645 | ||
2193837a LC |
1646 | 5.1.3alpha (2013-10-26) |
1647 | ||
1648 | * All fixes from 5.0.5 | |
1649 | ||
1650 | * liblzma: | |
1651 | ||
1652 | - Fixed a deadlock in the threaded encoder. | |
1653 | ||
1654 | - Made the uses of lzma_allocator const correct. | |
1655 | ||
1656 | - Added lzma_block_uncomp_encode() to create uncompressed | |
1657 | .xz Blocks using LZMA2 uncompressed chunks. | |
1658 | ||
1659 | - Added support for native threads on Windows and the ability | |
1660 | to detect the number of CPU cores. | |
1661 | ||
1662 | * xz: | |
1663 | ||
1664 | - Fixed a race condition in the signal handling. It was | |
1665 | possible that e.g. the first SIGINT didn't make xz exit | |
1666 | if reading or writing blocked and one had bad luck. The fix | |
1667 | is non-trivial, so as of writing it is unknown if it will be | |
1668 | backported to the v5.0 branch. | |
1669 | ||
1670 | - Made the progress indicator work correctly in threaded mode. | |
1671 | ||
1672 | - Threaded encoder now works together with --block-list=SIZES. | |
1673 | ||
1674 | - Added preliminary support for --flush-timeout=TIMEOUT. | |
1675 | It can be useful for (somewhat) real-time streaming. For | |
1676 | now the decompression side has to be done with something | |
1677 | else than the xz tool due to how xz does buffering, but this | |
1678 | should be fixed. | |
1679 | ||
1680 | ||
8f3c1d88 LC |
1681 | 5.1.2alpha (2012-07-04) |
1682 | ||
1683 | * All fixes from 5.0.3 and 5.0.4 | |
1684 | ||
1685 | * liblzma: | |
1686 | ||
1687 | - Fixed a deadlock and an invalid free() in the threaded encoder. | |
1688 | ||
1689 | - Added support for symbol versioning. It is enabled by default | |
1690 | on GNU/Linux, other GNU-based systems, and FreeBSD. | |
1691 | ||
1692 | - Use SHA-256 implementation from the operating system if one is | |
1693 | available in libc, libmd, or libutil. liblzma won't use e.g. | |
1694 | OpenSSL or libgcrypt to avoid introducing new dependencies. | |
1695 | ||
1696 | - Fixed liblzma.pc for static linking. | |
1697 | ||
1698 | - Fixed a few portability bugs. | |
1699 | ||
1700 | * xz --decompress --single-stream now fixes the input position after | |
1701 | successful decompression. Now the following works: | |
1702 | ||
1703 | echo foo | xz > foo.xz | |
1704 | echo bar | xz >> foo.xz | |
1705 | ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz | |
1706 | ||
1707 | Note that it doesn't work if the input is not seekable | |
1708 | or if there is Stream Padding between the concatenated | |
1709 | .xz Streams. | |
1710 | ||
1711 | * xz -lvv now shows the minimum xz version that is required to | |
1712 | decompress the file. Currently it is 5.0.0 for all supported .xz | |
1713 | files except files with empty LZMA2 streams require 5.0.2. | |
1714 | ||
1715 | * Added an *incomplete* implementation of --block-list=SIZES to xz. | |
1716 | It only works correctly in single-threaded mode and when | |
1717 | --block-size isn't used at the same time. --block-list allows | |
1718 | specifying the sizes of Blocks which can be useful e.g. when | |
1719 | creating files for random-access reading. | |
1720 | ||
1721 | ||
d91a84b5 LC |
1722 | 5.1.1alpha (2011-04-12) |
1723 | ||
1724 | * All fixes from 5.0.2 | |
1725 | ||
1726 | * liblzma fixes that will also be included in 5.0.3: | |
1727 | ||
1728 | - A memory leak was fixed. | |
1729 | ||
1730 | - lzma_stream_buffer_encode() no longer creates an empty .xz | |
1731 | Block if encoding an empty buffer. Such an empty Block with | |
1732 | LZMA2 data would trigger a bug in 5.0.1 and older (see the | |
1733 | first bullet point in 5.0.2 notes). When releasing 5.0.2, | |
1734 | I thought that no encoder creates this kind of files but | |
1735 | I was wrong. | |
1736 | ||
1737 | - Validate function arguments better in a few functions. Most | |
1738 | importantly, specifying an unsupported integrity check to | |
1739 | lzma_stream_buffer_encode() no longer creates a corrupt .xz | |
1740 | file. Probably no application tries to do that, so this | |
1741 | shouldn't be a big problem in practice. | |
1742 | ||
1743 | - Document that lzma_block_buffer_encode(), | |
1744 | lzma_easy_buffer_encode(), lzma_stream_encoder(), and | |
1745 | lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. | |
1746 | ||
1747 | - The return values of the _memusage() functions are now | |
1748 | documented better. | |
1749 | ||
1750 | * Support for multithreaded compression was added using the simplest | |
1751 | method, which splits the input data into blocks and compresses | |
1752 | them independently. Other methods will be added in the future. | |
1753 | The current method has room for improvement, e.g. it is possible | |
1754 | to reduce the memory usage. | |
1755 | ||
1756 | * Added the options --single-stream and --block-size=SIZE to xz. | |
1757 | ||
1758 | * xzdiff and xzgrep now support .lzo files if lzop is installed. | |
1759 | The .tzo suffix is also recognized as a shorthand for .tar.lzo. | |
1760 | ||
1761 | * Support for short 8.3 filenames under DOS was added to xz. It is | |
1762 | experimental and may change before it gets into a stable release. | |
1763 | ||
6e1326fc | 1764 | |
cec2ee86 LC |
1765 | 5.0.8 (2014-12-21) |
1766 | ||
1767 | * Fixed an old bug in xzgrep that affected OpenBSD and probably | |
1768 | a few other operating systems too. | |
1769 | ||
1770 | * Updated French and German translations. | |
1771 | ||
1772 | * Added support for detecting the amount of RAM on AmigaOS/AROS. | |
1773 | ||
1774 | * Minor build system updates. | |
1775 | ||
1776 | ||
76e75522 LC |
1777 | 5.0.7 (2014-09-20) |
1778 | ||
1779 | * Fix regressions introduced in 5.0.6: | |
1780 | ||
1781 | - Fix building with non-GNU make. | |
1782 | ||
1783 | - Fix invalid Libs.private value in liblzma.pc which broke | |
1784 | static linking against liblzma if the linker flags were | |
1785 | taken from pkg-config. | |
1786 | ||
1787 | ||
e9e097e2 LC |
1788 | 5.0.6 (2014-09-14) |
1789 | ||
1790 | * xzgrep now exits with status 0 if at least one file matched. | |
1791 | ||
1792 | * A few minor portability and build system fixes | |
1793 | ||
1794 | ||
1936718b LC |
1795 | 5.0.5 (2013-06-30) |
1796 | ||
1797 | * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing | |
1798 | .lzma files that have less common settings in the headers | |
1799 | (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed | |
1800 | size greater than 256 GiB). The limitations existed to avoid false | |
1801 | positives when detecting .lzma files. The lc + lp <= 4 limitation | |
1802 | still remains since liblzma's LZMA decoder has that limitation. | |
1803 | ||
1804 | NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT | |
1805 | affected by this change. They still consider uncommon .lzma headers | |
1806 | as not being in the .lzma format. Changing this would give way too | |
1807 | many false positives. | |
1808 | ||
1809 | * xz: | |
1810 | ||
1811 | - Interaction of preset and custom filter chain options was | |
1812 | made less illogical. This affects only certain less typical | |
1813 | uses cases so few people are expected to notice this change. | |
1814 | ||
1815 | Now when a custom filter chain option (e.g. --lzma2) is | |
1816 | specified, all preset options (-0 ... -9, -e) earlier are on | |
1817 | the command line are completely forgotten. Similarly, when | |
1818 | a preset option is specified, all custom filter chain options | |
1819 | earlier on the command line are completely forgotten. | |
1820 | ||
1821 | Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e" | |
1822 | which is equivalent to "xz -6e". Earlier -e didn't put xz back | |
1823 | into preset mode and thus the example command was equivalent | |
1824 | to "xz --lzma2=preset=5". | |
1825 | ||
1826 | Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to | |
1827 | "xz -7". Earlier a custom filter chain option didn't make | |
1828 | xz forget the -e option so the example was equivalent to | |
1829 | "xz -7e". | |
1830 | ||
1831 | - Fixes and improvements to error handling. | |
1832 | ||
1833 | - Various fixes to the man page. | |
1834 | ||
1835 | * xzless: Fixed to work with "less" versions 448 and later. | |
1836 | ||
1837 | * xzgrep: Made -h an alias for --no-filename. | |
1838 | ||
1839 | * Include the previously missing debug/translation.bash which can | |
1840 | be useful for translators. | |
1841 | ||
1842 | * Include a build script for Mac OS X. This has been in the Git | |
1843 | repository since 2010 but due to a mistake in Makefile.am the | |
1844 | script hasn't been included in a release tarball before. | |
1845 | ||
1846 | ||
eccd8017 LC |
1847 | 5.0.4 (2012-06-22) |
1848 | ||
1849 | * liblzma: | |
1850 | ||
1851 | - Fix lzma_index_init(). It could crash if memory allocation | |
1852 | failed. | |
1853 | ||
1854 | - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ | |
1855 | filter is used and the application only provides exactly as | |
1856 | much output space as is the uncompressed size of the file. | |
1857 | ||
1858 | - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't | |
1859 | check if the last call to lzma_code() really returned | |
1860 | LZMA_STREAM_END, which made the program think that truncated | |
1861 | files are valid. | |
1862 | ||
1863 | - New example programs in doc/examples (old programs are now in | |
1864 | doc/examples_old). These have more comments and more detailed | |
1865 | error handling. | |
1866 | ||
1867 | * Fix "xz -lvv foo.xz". It could crash on some corrupted files. | |
1868 | ||
1869 | * Fix output of "xz --robot -lv" and "xz --robot -lvv" which | |
1870 | incorrectly printed the filename also in the "foo (x/x)" format. | |
1871 | ||
1872 | * Fix exit status of "xzdiff foo.xz bar.xz". | |
1873 | ||
1874 | * Fix exit status of "xzgrep foo binary_file". | |
1875 | ||
1876 | * Fix portability to EBCDIC systems. | |
1877 | ||
1878 | * Fix a configure issue on AIX with the XL C compiler. See INSTALL | |
1879 | for details. | |
1880 | ||
1881 | * Update French, German, Italian, and Polish translations. | |
1882 | ||
1883 | ||
48053e8a LC |
1884 | 5.0.3 (2011-05-21) |
1885 | ||
1886 | * liblzma fixes: | |
1887 | ||
1888 | - A memory leak was fixed. | |
1889 | ||
1890 | - lzma_stream_buffer_encode() no longer creates an empty .xz | |
1891 | Block if encoding an empty buffer. Such an empty Block with | |
1892 | LZMA2 data would trigger a bug in 5.0.1 and older (see the | |
1893 | first bullet point in 5.0.2 notes). When releasing 5.0.2, | |
1894 | I thought that no encoder creates this kind of files but | |
1895 | I was wrong. | |
1896 | ||
1897 | - Validate function arguments better in a few functions. Most | |
1898 | importantly, specifying an unsupported integrity check to | |
1899 | lzma_stream_buffer_encode() no longer creates a corrupt .xz | |
1900 | file. Probably no application tries to do that, so this | |
1901 | shouldn't be a big problem in practice. | |
1902 | ||
1903 | - Document that lzma_block_buffer_encode(), | |
1904 | lzma_easy_buffer_encode(), lzma_stream_encoder(), and | |
1905 | lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. | |
1906 | ||
1907 | - The return values of the _memusage() functions are now | |
1908 | documented better. | |
1909 | ||
1910 | * Fix command name detection in xzgrep. xzegrep and xzfgrep now | |
1911 | correctly use egrep and fgrep instead of grep. | |
1912 | ||
1913 | * French translation was added. | |
1914 | ||
1915 | ||
16889013 LC |
1916 | 5.0.2 (2011-04-01) |
1917 | ||
1918 | * LZMA2 decompressor now correctly accepts LZMA2 streams with no | |
1919 | uncompressed data. Previously it considered them corrupt. The | |
1920 | bug can affect applications that use raw LZMA2 streams. It is | |
1921 | very unlikely to affect .xz files because no compressor creates | |
1922 | .xz files with empty LZMA2 streams. (Empty .xz files are a | |
1923 | different thing than empty LZMA2 streams.) | |
1924 | ||
1925 | * "xz --suffix=.foo filename.foo" now refuses to compress the | |
1926 | file due to it already having the suffix .foo. It was already | |
1927 | documented on the man page, but the code lacked the test. | |
1928 | ||
1929 | * "xzgrep -l foo bar.xz" works now. | |
1930 | ||
1931 | * Polish translation was added. | |
1932 | ||
1933 | ||
5fbce0b8 LC |
1934 | 5.0.1 (2011-01-29) |
1935 | ||
1936 | * xz --force now (de)compresses files that have setuid, setgid, | |
1937 | or sticky bit set and files that have multiple hard links. | |
1938 | The man page had it documented this way already, but the code | |
1939 | had a bug. | |
1940 | ||
1941 | * gzip and bzip2 support in xzdiff was fixed. | |
1942 | ||
1943 | * Portability fixes | |
1944 | ||
1945 | * Minor fix to Czech translation | |
1946 | ||
1947 | ||
6e1326fc LC |
1948 | 5.0.0 (2010-10-23) |
1949 | ||
1950 | Only the most important changes compared to 4.999.9beta are listed | |
1951 | here. One change is especially important: | |
1952 | ||
1953 | * The memory usage limit is now disabled by default. Some scripts | |
1954 | written before this change may have used --memory=max on xz command | |
1955 | line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED | |
1956 | NOW, because they interfere with user's ability to set the memory | |
1957 | usage limit himself. If user-specified limit causes problems to | |
1958 | your script, blame the user. | |
1959 | ||
1960 | Other significant changes: | |
1961 | ||
1962 | * Added support for XZ_DEFAULTS environment variable. This variable | |
1963 | allows users to set default options for xz, e.g. default memory | |
1964 | usage limit or default compression level. Scripts that use xz | |
1965 | must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT | |
1966 | instead if they need a way to pass options to xz via an | |
1967 | environment variable. | |
1968 | ||
1969 | * The compression settings associated with the preset levels | |
1970 | -0 ... -9 have been changed. --extreme was changed a little too. | |
1971 | It is now less likely to make compression worse, but with some | |
1972 | files the new --extreme may compress slightly worse than the old | |
1973 | --extreme. | |
1974 | ||
1975 | * If a preset level (-0 ... -9) is specified after a custom filter | |
1976 | chain options have been used (e.g. --lzma2), the custom filter | |
1977 | chain will be forgotten. Earlier the preset options were | |
1978 | completely ignored after custom filter chain options had been | |
1979 | seen. | |
1980 | ||
1981 | * xz will create sparse files when decompressing if the uncompressed | |
1982 | data contains long sequences of binary zeros. This is done even | |
1983 | when writing to standard output that is connected to a regular | |
1984 | file and certain additional conditions are met to make it safe. | |
1985 | ||
1986 | * Support for "xz --list" was added. Combine with --verbose or | |
1987 | --verbose --verbose (-vv) for detailed output. | |
1988 | ||
1989 | * I had hoped that liblzma API would have been stable after | |
1990 | 4.999.9beta, but there have been a couple of changes in the | |
1991 | advanced features, which don't affect most applications: | |
1992 | ||
1993 | - Index handling code was revised. If you were using the old | |
1994 | API, you will get a compiler error (so it's easy to notice). | |
1995 | ||
1996 | - A subtle but important change was made to the Block handling | |
1997 | API. lzma_block.version has to be initialized even for | |
1998 | lzma_block_header_decode(). Code that doesn't do it will work | |
1999 | for now, but might break in the future, which makes this API | |
2000 | change easy to miss. | |
2001 | ||
2002 | * The major soname has been bumped to 5.0.0. liblzma API and ABI | |
2003 | are now stable, so the need to recompile programs linking against | |
2004 | liblzma shouldn't arise soon. | |
2005 |