thirdparty/e2fsprogs.git
10 hours agoe2fsck: correctly handle inline directories when large_dir is enabled. maint master next
Artem Blagodarenko [Fri, 14 Jun 2019 14:42:37 +0000 (17:42 +0300)]
e2fsck: correctly handle inline directories when large_dir is enabled.

Historically, e2fsck has required that directories not contain holes.
(In fact, as of this writing, ext4 still requires this to be the
case.)  Commit ae9efd05a98 ("e2fsck: 3 level hash tree directory
optimization") removed this requirement if the large_dir feature is
enabled; however, the way it was done caused it to incorrectly handle
inline directories.

To reproduce the problem fixed by this commit:

truncate -s 100000000 ext4.img
misc/mke2fs -t ext4 -I 512 -O 'inline_data,large_dir' ext4.img
mkdir m
sudo mount ext4.img m
mkdir m/aa
sudo umount m
e2fsck/e2fsck -f -n ext4.img

The last command gives this output:

[root@localhost e2fsprogs-kernel]# e2fsck/e2fsck -f -n ext4-2.img
e2fsck 1.45.2 (27-May-2019)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
'..' in /aa (12) is <The NULL inode> (0), should be / (2).
Fix? no

Pass 4: Checking reference counts
Inode 2 ref count is 4, should be 3.  Fix? no

Inode 12 ref count is 2, should be 1.  Fix? no

Pass 5: Checking group summary information

ext4-2.img: ********** WARNING: Filesystem still has errors **********

ext4-2.img: 12/24384 files (0.0% non-contiguous), 17874/97656 blocks

Reported-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 days agoFix posix_memalign and posix_fadvise calls.
Theodore Ts'o [Fri, 7 Jun 2019 17:07:12 +0000 (13:07 -0400)]
Fix posix_memalign and posix_fadvise calls.

Almost all posix_ functions return a positive errno value (without
setting errno) rather than -1 and setting errno. Most calls in this
project were correct, but these two weren't.

Reported-by: Hughes <enh@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 weeks agoe2scrub: remove -C from e2scrub_all
Darrick J. Wong [Tue, 4 Jun 2019 04:27:12 +0000 (21:27 -0700)]
e2scrub: remove -C from e2scrub_all

We already have the "SERVICE_MODE=1" feature that signals to e2scrub
that we're running as a background daemon and therefore we should exit
quietly if conditions aren't right.

It's therefore unnecessary to have a separate -C flag to achieve the
same outcome for cron jobs.  Merge the two together.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 weeks agopo: update pt.po (from translationproject.org)
Pedro Albuquerque [Thu, 30 May 2019 21:05:57 +0000 (17:05 -0400)]
po: update pt.po (from translationproject.org)

Note: This fixes the yes/no matching strings for mke2fs and tune2fs if
Portuguese is in use.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 weeks agotests: add test for e2fsck of verity file
Eric Biggers [Tue, 28 May 2019 23:12:30 +0000 (16:12 -0700)]
tests: add test for e2fsck of verity file

Test that e2fsck doesn't report errors when an inode with the 'verity'
flag has blocks past i_size.

This is a regression test for commits 3baafde6a8ae ("e2fsck: allow
verity files to have initialized blocks past i_size") and 43466d039689
("e2fsck: handle verity files in scan_extent_node()").

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agoUpdate release notes, etc., for the 1.45.2 release v1.45.2
Theodore Ts'o [Tue, 28 May 2019 02:10:51 +0000 (22:10 -0400)]
Update release notes, etc., for the 1.45.2 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agopo: update the binary gmo files
Theodore Ts'o [Tue, 28 May 2019 00:05:54 +0000 (20:05 -0400)]
po: update the binary gmo files

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agomke2fs: accept the english yes character to the proceed question
Theodore Ts'o [Mon, 27 May 2019 23:36:15 +0000 (19:36 -0400)]
mke2fs: accept the english yes character to the proceed question

In some cases if the translation file is missing some translations,
mke2fs can end up printing an English message, e.g.:

% LANG=it_IT.UTF-8 ./mke2fs /tmp/foo.img 8M
mke2fs 1.45.1 (12-May-2019)
/tmp/foo.img contiene un file system ext4
created on Mon May 27 19:35:48 2019
Proceed anyway? (y,N)

However, if there is a translation for string to match with "yY"
(e.g., to "sS" for Italian), then 'y' won't work.  Fix this by falling
back to the english 'yY' characters.

Addresses-Debian-Bug: #907034
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agopo: update ms.po (from translationproject.org)
Sharuzzaman Ahmat Raslan [Mon, 27 May 2019 04:24:01 +0000 (00:24 -0400)]
po: update ms.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agopo: update es.po (from translationproject.org)
Antonio Ceballos [Mon, 27 May 2019 04:24:01 +0000 (00:24 -0400)]
po: update es.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agoe2fsck: handle verity files in scan_extent_node()
Eric Biggers [Thu, 23 May 2019 15:30:33 +0000 (08:30 -0700)]
e2fsck: handle verity files in scan_extent_node()

Don't report PR_1_EXTENT_END_OUT_OF_BOUNDS on verity files during
scan_extent_node(), since they will have blocks stored past i_size.

This was missed during the earlier fix because this check only triggers
if the inode has enough extents to need at least one extent index node.

This bug is causing one of the fs-verity xfstests to fail with the
reworked fs-verity patchset.

Fixes: 3baafde6a8ae ("e2fsck: allow verity files to have initialized blocks past i_size")
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agoconfigure: fix test where cron is not installed on a non-systemd system
Theodore Ts'o [Wed, 22 May 2019 17:03:38 +0000 (13:03 -0400)]
configure: fix test where cron is not installed on a non-systemd system

Reported-by: thomas@linuxfromscratch.org
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 weeks agopo: add new Portuguese translation from the Translation Project
Theodore Ts'o [Wed, 22 May 2019 16:33:55 +0000 (12:33 -0400)]
po: add new Portuguese translation from the Translation Project

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agopo: update sv.po (from translationproject.org)
Göran Uddeborg [Wed, 22 May 2019 16:31:37 +0000 (12:31 -0400)]
po: update sv.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agodebian: update changelog for maintenance releases
Theodore Ts'o [Wed, 22 May 2019 16:31:04 +0000 (12:31 -0400)]
debian: update changelog for maintenance releases

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agoe2scrub_all: fix missing getopts argument which broke e2scrub_all -C
Theodore Ts'o [Tue, 21 May 2019 00:34:59 +0000 (20:34 -0400)]
e2scrub_all: fix missing getopts argument which broke e2scrub_all -C

Addresses-Debian-Bug: #929287

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agoe2scrub: fix grammar nit: "a LVM" -> "an LVM"
Theodore Ts'o [Mon, 20 May 2019 14:56:51 +0000 (10:56 -0400)]
e2scrub: fix grammar nit: "a LVM" -> "an LVM"

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agopo: update uk.po (from translationproject.org)
Yuri Chornoivan [Sun, 19 May 2019 03:32:30 +0000 (23:32 -0400)]
po: update uk.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agopo: update pl.po (from translationproject.org)
Jakub Bogusz [Sun, 19 May 2019 03:32:29 +0000 (23:32 -0400)]
po: update pl.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agopo: update cs.po (from translationproject.org)
Petr Pisar [Sun, 19 May 2019 03:32:29 +0000 (23:32 -0400)]
po: update cs.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agoe2scrub_all: avoid scrubbing all devices when there is nothing to scrub
Theodore Ts'o [Sun, 19 May 2019 03:04:49 +0000 (23:04 -0400)]
e2scrub_all: avoid scrubbing all devices when there is nothing to scrub

Running lsblk when there are no valid block devicse results in
generating all block devices as the list of devices to scrub; this
results in a lot of e2scrub_all failures.

Addresses-Debian-Bug: #929186

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 weeks agoe2scrub: stop cron spam if lvm2 is not installed.
Theodore Ts'o [Thu, 16 May 2019 18:56:37 +0000 (14:56 -0400)]
e2scrub: stop cron spam if lvm2 is not installed.

Addresses-Debian-Bug: #928977

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agoUpdate release notes, etc., for the 1.45.1 release v1.45.1
Theodore Ts'o [Mon, 13 May 2019 01:10:29 +0000 (21:10 -0400)]
Update release notes, etc., for the 1.45.1 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agoe2fsck: remove an potentially ambiguous dangling else clause
Theodore Ts'o [Fri, 10 May 2019 23:21:03 +0000 (19:21 -0400)]
e2fsck: remove an potentially ambiguous dangling else clause

This doesn't actually fix a bug or change behavior, but it removes a
clang warning.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agolibext2fs: hide struct ext2fs_hashmap as an internal implementation detail
Theodore Ts'o [Fri, 10 May 2019 23:00:13 +0000 (19:00 -0400)]
libext2fs: hide struct ext2fs_hashmap as an internal implementation detail

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agomke2fs: fix memory leak when parsing encoding flags
Theodore Ts'o [Fri, 10 May 2019 18:38:24 +0000 (14:38 -0400)]
mke2fs: fix memory leak when parsing encoding flags

Also fix extended-options usage string.

Addresses-Coverity-Bug: 1444984
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agoe2fsck: fix printf format / argument mismatches
Theodore Ts'o [Fri, 10 May 2019 18:26:27 +0000 (14:26 -0400)]
e2fsck: fix printf format / argument mismatches

Fixes-Coverity-Bug: 1444982
Fixes-Coverity-Bug: 1444983
Fixes-Coverity-Bug: 1444985
Fixes-Coverity-Bug: 1444986
Fixes-Coverity-Bug: 1444987
Fixes-Coverity-Bug: 1444988
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agolibext2fs: add missing check for utf8lookup()'s return value
Theodore Ts'o [Fri, 10 May 2019 18:18:12 +0000 (14:18 -0400)]
libext2fs: add missing check for utf8lookup()'s return value

Fixes-Coverity-Bug: 1442630
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.com>
5 weeks agoFix parallel install issue in scrub subdir
Michael Haubenwallner [Tue, 7 May 2019 12:49:25 +0000 (14:49 +0200)]
Fix parallel install issue in scrub subdir

In scrub/Makefile the various 'install-*' targets do not explicitly
depend on their corresponding 'installdirs-*' target, so they get run
in parallel.

Addresses-Gentoo-Bug: #680030

Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
5 weeks agoe2fsck: fix XML validity problem with the problem log
Theodore Ts'o [Fri, 10 May 2019 17:20:59 +0000 (13:20 -0400)]
e2fsck: fix XML validity problem with the problem log

Also fix a gcc warning.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agoUpdate release notes, etc., for the 1.45.1-rc1 release v1.45.1-rc1
Theodore Ts'o [Mon, 6 May 2019 23:12:58 +0000 (19:12 -0400)]
Update release notes, etc., for the 1.45.1-rc1 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agoe2fsck: add support for saving the problem code log
Theodore Ts'o [Mon, 6 May 2019 05:10:53 +0000 (01:10 -0400)]
e2fsck: add support for saving the problem code log

Add the ability to save a log of problems found by e2fsck in a log
file that can be specified via /etc/e2fsck.conf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agoe2scrub: make the e2scrub service files require CAP_SYS_ADMIN and CAP_SYS_RAWIO
Theodore Ts'o [Mon, 6 May 2019 02:26:04 +0000 (22:26 -0400)]
e2scrub: make the e2scrub service files require CAP_SYS_ADMIN and CAP_SYS_RAWIO

Addresses-Debian-Bug: #926138

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agoe2scrub: tag the *.e2scrub LV's with UDISK_IGNORE in udev
Theodore Ts'o [Mon, 6 May 2019 02:00:16 +0000 (22:00 -0400)]
e2scrub: tag the *.e2scrub LV's with UDISK_IGNORE in udev

Force the *.e2scrub LV's to be ignored by udisk.

Addresses-Debian-Bug: #926112

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agomke2fs: fix check for absurdly large devices
Andreas Dilger [Sun, 5 May 2019 22:33:46 +0000 (18:33 -0400)]
mke2fs: fix check for absurdly large devices

The check in mke2fs is intended to be for the number of blocks in the
filesystem exceeding the maximum number of addressable blocks in 2^32
bitmaps, which is (2^32 * 8 bits/byte * blocksize) = 2^47 blocks,
or 2^59 bytes = 512PiB for the common 4KiB blocksize.

However, s_log_blocksize holds log2(blocksize_in_kb), so the current
calculation is a factor of 2^10 too small.  This caused mke2fs to fail
while trying to format a 900TB filesystem.

Fixes: 101ef2e93c25 ("mke2fs: Avoid crashes / infinite loops for absurdly large devices")
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agoe2fsck: check and fix tails of all bitmap blocks
Theodore Ts'o [Sun, 5 May 2019 20:43:33 +0000 (16:43 -0400)]
e2fsck: check and fix tails of all bitmap blocks

Currently, e2fsck effectively checks only tail of the last inode and
block bitmap in the filesystem. Thus if some previous bitmap has unset
bits it goes unnoticed.  Mostly these tail bits in the bitmap are
ignored; however, if blocks_per_group are smaller than 8*blocksize,
the multi-block allocator in the kernel can get confused when the tail
bits are unset and return bogus free extent.

Add support to libext2fs to check these bitmap tails when loading
bitmaps (as that's about the only place which has access to the bitmap
tail bits) and make e2fsck use this functionality to detect buggy bitmap
tails and fix them (by rewriting the bitmaps).

Reported-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
6 weeks agodebian: update libext2fs2.symbols file
Theodore Ts'o [Sun, 5 May 2019 02:51:43 +0000 (22:51 -0400)]
debian: update libext2fs2.symbols file

Update to reflect changes to avoid leaking utf8 and nls symbols into
the namespace.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agolibext2fs: move struct ext2fs_nls_table to the private ext2fsP.h header
Theodore Ts'o [Fri, 3 May 2019 20:42:36 +0000 (16:42 -0400)]
libext2fs: move struct ext2fs_nls_table to the private ext2fsP.h header

Callers of libext2fs don't need to use this structure, and this gives
us the ability to change it later on without worrying about
changing public ABI's.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agoRename the feature "fname_encoding" to be "casefold".
Theodore Ts'o [Fri, 3 May 2019 17:16:29 +0000 (13:16 -0400)]
Rename the feature "fname_encoding" to be "casefold".

Also change mke2fs so that the encoding and encoding flags are
specified in mke2fs.conf in the fs_types and defaults stanzas instead
of the options stanza.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agodebugfs: teach set_super_value how to set the s_encoding field
Theodore Ts'o [Thu, 2 May 2019 17:19:01 +0000 (13:19 -0400)]
debugfs: teach set_super_value how to set the s_encoding field

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
6 weeks agolibe2p: print the filename character encoding in list_super[2]
Theodore Ts'o [Thu, 2 May 2019 15:48:45 +0000 (11:48 -0400)]
libe2p: print the filename character encoding in list_super[2]

Also clean up some gcc -Wall warnings while we're at it.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agolibext2fs: remove utf8_* namespace contamination
Theodore Ts'o [Mon, 29 Apr 2019 13:15:48 +0000 (09:15 -0400)]
libext2fs: remove utf8_* namespace contamination

Merge nls_utf8-norm.c and nls_utf8.c.  This also allows us to comment
out functions which we don't actually need for e2fsprogs.

Also fix some gcc -Wall complaints, including one which would have
caused utf8_casefold() to misbehave (this was fixed in the kernel, but
not carried back to e2fsprogs).

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agolibext2fs: remove nls_* namespace contamination
Theodore Ts'o [Mon, 29 Apr 2019 02:04:10 +0000 (22:04 -0400)]
libext2fs: remove nls_* namespace contamination

Also remove nls.h, and avoid declaring static functions and variables
in a header file, which is wasteful of space.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agolibext2fs: remove unused variable 'old_flags'
Eric Biggers [Mon, 29 Apr 2019 00:37:45 +0000 (20:37 -0400)]
libext2fs: remove unused variable 'old_flags'

In ext2fs_write_inode2(), the 'old_flags' variable is never used.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agodebugfs: remove unused variable 'tmp_inode'
Eric Biggers [Mon, 29 Apr 2019 00:37:21 +0000 (20:37 -0400)]
debugfs: remove unused variable 'tmp_inode'

In parse_inode_csum(), the outer 'tmp_inode' variable is never used.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agolibext2fs: remove unused variable 'buff'
Eric Biggers [Mon, 22 Apr 2019 21:08:43 +0000 (14:08 -0700)]
libext2fs: remove unused variable 'buff'

In ext2fs_dirhash2(), the outer 'buff' variable is never used.

Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agodebugfs: fix encoding handling in dx_hash command
Eric Biggers [Mon, 29 Apr 2019 00:35:21 +0000 (20:35 -0400)]
debugfs: fix encoding handling in dx_hash command

Fix the following bugs:

1. 'encoding' and 'hash_flags' are not initialized, causing a segfault.

2. 'hash_flags' incorrectly uses a __bitwise type.

3. The optstring doesn't contain "c" or "e", so the -c and -e options
   aren't recognized.

4. The code that handles the -e option always returns.

Fixes: ef733f1a97ec ("debugfs: support encoding when printing the file hash")
Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agodebugfs: avoid ambiguity when printing filenames
Eric Biggers [Sun, 28 Apr 2019 23:42:41 +0000 (19:42 -0400)]
debugfs: avoid ambiguity when printing filenames

The way debugfs escapes filenames is ambiguous because a sequence like
M-A can mean either the byte 'A' + 128 == 0xc1 or the three bytes
{'M', '-', 'A'}.  Similarly, ^A can mean either the byte
'A' ^ 0x40 == 0x01 or the two bytes {'^', 'A'}.

Fix this and simplify the code by switching to a simpler strategy where
all bytes < 32, all bytes >= 127, and backslash are encoded with C-style
hex escape sequences.  E.g., the byte 0xc1 will now be encoded as \xc1
rather than M-A as it was before, while a filename consisting of the
three bytes {'M', '-', 'A'} will continue to be shown as M-A.

I want to fix this mainly because I want to use debugfs to retrieve raw
encrypted filenames for ciphertext verification tests.  But this doesn't
work if the returned filenames are ambiguous.

Fixes: 68a1de3df340 ("debugfs: pretty print encrypted filenames in the ls command")
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext4.5.in: document design changes on the casefold attribute
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 20:53:08 +0000 (16:53 -0400)]
ext4.5.in: document design changes on the casefold attribute

Document the fact that the encoding support is only used by directories
with the +F attribute.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: drop ASCII NLS support
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 20:52:12 +0000 (16:52 -0400)]
ext2fs: drop ASCII NLS support

As agreed on the kernel side, ext4 will only support utf8 for now, and
not plain ASCII, so we don't need this anymore.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: drop Unicode normalization support
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 20:48:01 +0000 (16:48 -0400)]
ext2fs: drop Unicode normalization support

No longer necessary after +F design change.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: don't normalize names on -F directories
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 20:45:41 +0000 (16:45 -0400)]
ext2fs: don't normalize names on -F directories

As agreed on the new design, even if fname_encoding is enabled,
directories entries who aren't owned by a +F directory will not be
normalized.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: nls: Update to unicode 12.1.0
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 20:33:36 +0000 (16:33 -0400)]
ext2fs: nls: Update to unicode 12.1.0

Since we didn't release a kernel version that supports version 11.0.0,
it should be safe to reuse the sb entry for version 12.1.0.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: merge sparse fixes for unicode normalization
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 19:12:12 +0000 (15:12 -0400)]
ext2fs: merge sparse fixes for unicode normalization

The sparse script complained about the following issues in the kernel
version of this script.  This patch port the fixes to the userspace
version.

sparse warnings: (new ones prefixed by >>)

>> fs/unicode/utf8-norm.c:41:24: sparse: non-ANSI function declaration
   of function 'utf8version_latest'

vim +/utf8version_latest +41 fs/unicode/utf8-norm.c

  40
> 41 int utf8version_latest()
  42 {
  43 return utf8vers;
  44 }
  45 EXPORT_SYMBOL(utf8version_latest);
  46

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: fix potential null dereference in utf8nlookup
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 19:10:21 +0000 (15:10 -0400)]
ext2fs: fix potential null dereference in utf8nlookup

Delay the access of data->offset until after the null check.  This was
reported by 0-day on the kernel version of the script.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: convert unicode normalization from NFKD -> NFD
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 19:08:39 +0000 (15:08 -0400)]
ext2fs: convert unicode normalization from NFKD -> NFD

Following the kernel transition, convert the normalization format from
NFKD to NFD.  This also regenerates the utf8data.h database.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoext2fs: always attempt to load nls table when loading the filesystem
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 19:06:27 +0000 (15:06 -0400)]
ext2fs: always attempt to load nls table when loading the filesystem

fs->encoding is exposed by the library, so we need to at least try to
load it when populating ext2_filsys.  Nevertheless, failing to do so
shouldn't be a fatal error, unless the user really needs that
information.  Thus, we ignore this failure during open/initialization
and let the user who needs it validate that field before trying to use
it.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 weeks agoe2p: encoding: fix default flags lookup
Gabriel Krisman Bertazi [Sun, 28 Apr 2019 19:04:28 +0000 (15:04 -0400)]
e2p: encoding: fix default flags lookup

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
8 weeks agoAOSP: Turn off some more warnings on Windows.
Elliott Hughes [Fri, 19 Apr 2019 16:34:39 +0000 (09:34 -0700)]
AOSP: Turn off some more warnings on Windows.

Test: builds
Change-Id: Ieb0f34ba444f608ec3fdf89aa61a93b4ac312af4
From AOSP commit: 88a90b52f836bc4aaba231857f1e4facf0a25959

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agodebugfs: fix printing of xattrs with ea_in_inode values
Theodore Ts'o [Tue, 26 Mar 2019 13:36:53 +0000 (09:36 -0400)]
debugfs: fix printing of xattrs with ea_in_inode values

Due to a missing "else" debugfs was printing (garbage) from the xattr
buffer which could potentially overrun the end of the buffer.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agotests: fix f_valid_ea_in_inode to use the ea_feature
Theodore Ts'o [Sun, 24 Mar 2019 06:36:08 +0000 (02:36 -0400)]
tests: fix f_valid_ea_in_inode to use the ea_feature

This time for real!  The test-ea file has three very long xattrs,
description and description2 which are identical and share an inode to
store their value, and description3, which has a different value, and
thus uses another inode for storing its value.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agoe2scrub,e2scrub_all: print a (more understandable) error if not run as root
Theodore Ts'o [Thu, 21 Mar 2019 01:58:52 +0000 (21:58 -0400)]
e2scrub,e2scrub_all: print a (more understandable) error if not run as root

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agoe2scrub_all: refactor device probe loop
Darrick J. Wong [Tue, 19 Mar 2019 00:17:32 +0000 (17:17 -0700)]
e2scrub_all: refactor device probe loop

Paul Menzel reported that the e2scrub_all reaper service that runs at
startup takes a long time to run, and Ted Ts'o pointed out that we could
do a lot less work by using lvs as the outer loop in the ext4 filesystem
probe function so that we only have to lsblk the lvm devices containing
ext4 filesystems.

Therefore, refactor the loops to put lvs first, which should boost speed
a bit.

[ Made some of the further optimizations suggested by Lukas Czerner.  -- TYT ]

Reported-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agoe2scrub_all: make sure there's enough free space for a snapshot
Theodore Ts'o [Wed, 20 Mar 2019 15:58:58 +0000 (11:58 -0400)]
e2scrub_all: make sure there's enough free space for a snapshot

If there isn't, skip the volume so we don't spam the system
administrator with error messages.  It's quite commkon that there is
is zero free space in the volume group.

Addresses-Debian-Bug: #924301

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agoe2scrub_all: add the -n option which shows what e2scrub_all would do
Theodore Ts'o [Mon, 18 Mar 2019 02:11:25 +0000 (22:11 -0400)]
e2scrub_all: add the -n option which shows what e2scrub_all would do

Also fix the copyright symbol so it is properly formatted when
processed into postscript or pdf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2 months agoe2scrub: add the -n option which shows what commands e2scrub would execute
Theodore Ts'o [Mon, 18 Mar 2019 01:15:46 +0000 (21:15 -0400)]
e2scrub: add the -n option which shows what commands e2scrub would execute

Also fix the copyright symbol so it is properly formatted when
processed into postscript or pdf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoe2scrub: fix up "make install-strip" support
Theodore Ts'o [Sun, 17 Mar 2019 22:50:37 +0000 (18:50 -0400)]
e2scrub: fix up "make install-strip" support

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoMerge branch 'old-maint' into maint
Theodore Ts'o [Sun, 17 Mar 2019 22:50:20 +0000 (18:50 -0400)]
Merge branch 'old-maint' into maint

3 months agoFix "make install-strip" old-maint
Theodore Ts'o [Sun, 17 Mar 2019 22:46:26 +0000 (18:46 -0400)]
Fix "make install-strip"

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agodebian: drop lvm2 from the recommends line
Theodore Ts'o [Mon, 11 Mar 2019 00:01:58 +0000 (20:01 -0400)]
debian: drop lvm2 from the recommends line

If the user doesn't intend to use lvm2, and it's not installed,
installing e2fsprogs shouldn't drag it (and all of its dependencies)
into the system.

Addresses-Debian-Bug: 924275

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoe2scrub: check to make sure lvm2 is installed
Theodore Ts'o [Sun, 10 Mar 2019 23:41:11 +0000 (19:41 -0400)]
e2scrub: check to make sure lvm2 is installed

Not all systems will have the lvm2 package installed, so check for
that.  Pretty much all systems should have util-linux installed, but
check for that as well.

Of course, if lvm2 is installed we shouldn't find any LVM devices ---
but eventually the Demon Murphy will find a way to make it happen. :-)

Also, set the PATH so we don't have to worry about the script failing
due to /sbin not being in the path.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoMerge branch 'old-maint' into maint
Theodore Ts'o [Thu, 7 Mar 2019 21:18:10 +0000 (16:18 -0500)]
Merge branch 'old-maint' into maint

3 months agoFix autoheader warnings
Dongyang Li [Wed, 6 Mar 2019 22:59:55 +0000 (22:59 +0000)]
Fix autoheader warnings

autoheader complains after 1.44.6 release:
autoheader: warning: missing template: HAVE_BLKID_PROBE_ENABLE_PARTITIONS
autoheader: Use AC_DEFINE([HAVE_BLKID_PROBE_ENABLE_PARTITIONS], [], [Description])
autoheader: warning: missing template: HAVE_BLKID_PROBE_GET_TOPOLOGY

This could fail automatic rpm builds.
Fixes: 7154d97 ("Check for the newer blkid functions without adding blkid to @LIBS@")

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoUpdate release notes, etc., for the 1.45.0 release v1.45.0
Theodore Ts'o [Wed, 6 Mar 2019 18:23:53 +0000 (13:23 -0500)]
Update release notes, etc., for the 1.45.0 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoe2scrub: install the crontab file in /etc/cron.d w/o the .cron extension
Theodore Ts'o [Thu, 7 Mar 2019 05:38:52 +0000 (00:38 -0500)]
e2scrub: install the crontab file in /etc/cron.d w/o the .cron extension

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoe2scrub: make e2scrub_fail's e-mail addresses be configurable
Theodore Ts'o [Thu, 7 Mar 2019 04:46:20 +0000 (23:46 -0500)]
e2scrub: make e2scrub_fail's e-mail addresses be configurable

Allow the sender and recipient e-mail addresses be configurable from
/etc/e2scrub.conf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoe2scrub: add missing Documentation links to systemd unit files
Theodore Ts'o [Thu, 7 Mar 2019 02:27:12 +0000 (21:27 -0500)]
e2scrub: add missing Documentation links to systemd unit files

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agomisc: fix spelling error
Theodore Ts'o [Thu, 7 Mar 2019 02:17:10 +0000 (21:17 -0500)]
misc: fix spelling error

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoscrub/Makefile.in: install the udev rule and crontab file as data files
Theodore Ts'o [Thu, 7 Mar 2019 01:09:37 +0000 (20:09 -0500)]
scrub/Makefile.in: install the udev rule and crontab file as data files

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoDon't use a symlink for config/ltmain.sh
Theodore Ts'o [Thu, 7 Mar 2019 01:06:26 +0000 (20:06 -0500)]
Don't use a symlink for config/ltmain.sh

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoFix "make clean" so it removes all generated files in the scrub directory
Theodore Ts'o [Wed, 6 Mar 2019 18:18:57 +0000 (13:18 -0500)]
Fix "make clean" so it removes all generated files in the scrub directory

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoUpdate Makefile.in files using "make depend"
Theodore Ts'o [Wed, 6 Mar 2019 18:14:19 +0000 (13:14 -0500)]
Update Makefile.in files using "make depend"

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agotests: add test for e2image -b option
Artem Blagodarenko [Wed, 6 Mar 2019 16:57:00 +0000 (11:57 -0500)]
tests: add test for e2image -b option

The test makes raw image from partition with broken super block
and executes e2fsck.

Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoe2image: add -b and -B options to specify where to find the superblock
Artem Blagodarenko [Wed, 6 Mar 2019 16:52:13 +0000 (11:52 -0500)]
e2image: add -b and -B options to specify where to find the superblock

e2image has no ability to use superblock backup to copy metadata.
This feature can be useful if someone wants to make partition
image and fix it using e2fsck utility.

New -b option allows to pass superblock number, like e2fsck utility does.
e2image doesn't change primary superblock and store it as is, so
it can be fixed using e2fsck latter. Option -B allows setting
superblock size.

Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoMerge branch 'maint' into next
Theodore Ts'o [Wed, 6 Mar 2019 05:06:42 +0000 (00:06 -0500)]
Merge branch 'maint' into next

3 months agoUpdate release notes, etc., for the 1.44.6 release v1.44.6
Theodore Ts'o [Tue, 5 Mar 2019 19:54:16 +0000 (14:54 -0500)]
Update release notes, etc., for the 1.44.6 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoAOSP: misc: use scandir with alphasort instead of readdir for consistency
Yoshitaka Seto [Wed, 16 Jan 2019 08:47:48 +0000 (17:47 +0900)]
AOSP: misc: use scandir with alphasort instead of readdir for consistency

During the image generation, When using the e2fsdroid with src_dir,
then dir files are listed with readdir, which will not guarantee
the order across the machines/fielsystems. So instead using the
scandir with alphasort option to list the files in sorted order.

Bug: 122874817

Change-Id: I4b946c737319252b82c74a0e10360843503862a1
From AOSP commit: 4cbe059b48b46d6657e24e4cdef543da7537dba3

Reported-by: Yasuhiro Kubota <yasuhiro.1.kubota@sony.com>
Tested-by: Yasuhiro Kubota <yasuhiro.1.kubota@sony.com>
Signed-off-by: Vikram Dattu Thota <vikram.dattu@sony.com>
Signed-off-by: Takuya Ogawa <takuya.ogawa@sony.com>
Signed-off-by: Yoshitaka Seto <yoshitaka.seto@sony.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
3 months agoAOSP: Fix file offset overflow issue when file's size > 4G
Chen Lin Z [Mon, 10 Dec 2018 07:31:40 +0000 (15:31 +0800)]
AOSP: Fix file offset overflow issue when file's size > 4G

fs->blocksize is int(4 bytes), while data is off_t(8 bytes),
'data_blk = data & ~(fs->blocksize - 1)' will cause data_blk
lose high 4 bytes of data if data > 4G and it'll cause file
inconsistent when using -d option to populate ext4 image file.

[ This was also fixed upstream via 1eec7413677e: "create_inode: fix
  copying large files".  This commit is just to clean up
  whitespace/formatting issues. -- tytso ]

Signed-off-by: Chen Lin Z <lin.z.chen@intel.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
From AOSP commit: 999dd56f2586fadec7bfe846b8cb52c5e528248f

4 months agolibext2fs: revert "try to always use PUNCH_HOLE for unix_discard"
Darrick J. Wong [Thu, 14 Feb 2019 21:02:33 +0000 (16:02 -0500)]
libext2fs: revert "try to always use PUNCH_HOLE for unix_discard"

Revert bcca9876a3428c10417c660b78933e6e70e8a5f5, because
fallocate(PUNCH_HOLE) on block devices was changed to use zeroout
instead of discard shortly after block device fallocate was merged.
zeroout isn't necessarily a "drop storage" operation like discard is,
so we prefer to use that on block devices.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 months agocreate_inode: fix potential memory leak in path_append()
Lukas Czerner [Mon, 11 Feb 2019 17:00:10 +0000 (12:00 -0500)]
create_inode: fix potential memory leak in path_append()

If realloc() fails in path_append() we will lose a memory pointed to by
target->path.  Fix it.

path_append() is used by mke2fs and e2fsdroid.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 months agoe2freefrag: fix memory leak in scan_online()
Lukas Czerner [Mon, 11 Feb 2019 16:52:50 +0000 (11:52 -0500)]
e2freefrag: fix memory leak in scan_online()

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 months agoMerge branch 'maint' into next
Theodore Ts'o [Mon, 11 Feb 2019 02:50:26 +0000 (21:50 -0500)]
Merge branch 'maint' into next

4 months agoCheck for the newer blkid functions without adding blkid to @LIBS@
Theodore Ts'o [Mon, 11 Feb 2019 02:44:05 +0000 (21:44 -0500)]
Check for the newer blkid functions without adding blkid to @LIBS@

If we are using the system-provided blkid, check for the newer blkid
functions without adding -lblkid to @LIBS@.  This prevents programs
(like badblocks) which don't use the blkid library getting linked
against it.

Addresses-Debian-Bug: #919958
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 months agoAOSP: E2fsprogs: fix ODR issue
Andreas Gampe [Wed, 30 Jan 2019 02:07:55 +0000 (18:07 -0800)]
AOSP: E2fsprogs: fix ODR issue

Do not use whole_static_libs and shared_libs at the same time. Fix
up clients.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bug: 123590866
Test: mmma external/e2fsprogs
Change-Id: I36d184dd878de2beacbb17c306e47076b44d68bb
From AOSP commit: fece156b105719c782351e447298de073d4dab04

4 months agoAOSP: e2fsdroid: Fix incorrect error value handling.
David Anderson [Thu, 24 Jan 2019 02:49:23 +0000 (18:49 -0800)]
AOSP: e2fsdroid: Fix incorrect error value handling.

Methods returning 0 or -1 should not pass their return value to com_err.
Also, errno values should not be negated when passed to com_err or
through errcode_t.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bug: N/A
Test: manual test

Change-Id: I5e25cf2deeee72668bf0ab58337ee582bf14bbc1
From AOSP commit: 0468e33fba75dbf60877ef14ecee7b0b60f756df

4 months agoAOSP: Build filefrag
Sandeep Patil [Wed, 26 Dec 2018 23:10:24 +0000 (15:10 -0800)]
AOSP: Build filefrag

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bug: 121211685
Test: mma -j with lunch aosp_blueline-userdebug

Change-Id: If8cffae8168990a11e8228b9f00d2fee15924935
Signed-off-by: Sandeep Patil <sspatil@google.com>
From AOSP commit: 0393dd541c79dfda1475ef071a31215dd2811162

4 months agocreate_inode: fix copying large files
Ross Burton [Thu, 7 Feb 2019 16:05:13 +0000 (11:05 -0500)]
create_inode: fix copying large files

When copying large files into a ext filesystem at mkfs time the copy
fails at 2^31 bytes in.  There are two problems:

copy_file_chunk() passes an offset (off_t, 64-bit typically) to
ext2fs_file_lseek() which expects a ext2_off_t (typedef to __u32) so
the value is truncated. Solve by calling ext2fs_file_llseek() which
takes a u64 offset instead.

try_lseek_copy() rounds the data and hole offsets as found by lseek()
to block boundaries, but the calculation gets truncated to 32-bits.
Solve by casting the 32-bit blocksize to off_t to ensure this doesn't
happen.

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
4 months agodebugfs: output large directory size
Artem Blagodarenko [Tue, 5 Feb 2019 22:10:05 +0000 (17:10 -0500)]
debugfs: output large directory size

large_dir option allows to create directory with size > 2GB.
debugfs utility outputs negative size value for large directories
and ignores high part of directory size.

Fix debugfs to use high part of directory size in debugfs output
and use appropriate output format.

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
4 months agoe2fsck: allow to check >2GB sized directory
Artem Blagodarenko [Tue, 5 Feb 2019 22:09:16 +0000 (17:09 -0500)]
e2fsck: allow to check >2GB sized directory

After large_dir feature has been added, e2fsprogs is
ready for directories > 2GB, so we can remove e2fsck
directory size check.

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
4 months agoMerge branch 'maint' into next
Theodore Ts'o [Wed, 30 Jan 2019 04:17:08 +0000 (23:17 -0500)]
Merge branch 'maint' into next