Theodore Ts'o [Sun, 16 Dec 2018 03:13:41 +0000 (22:13 -0500)]
e2fsck: use "Optimize?" prompt for PR_1E_CAN_{COLLAPSE,NARROW}_EXTENT_TREE
If e2fsck can optimize an inode's extent tree, it should ask for
permission to optimize it, not to "fix" it. This was causing some
confusion, since some users interpreted this prompt as an indication
that the file system was inconsistent.
Theodore Ts'o [Fri, 7 Dec 2018 03:44:45 +0000 (22:44 -0500)]
libext2fs: fix regression so we are correctly translating Posix ACL's
The commit 50d0998cfee ("libext2fs: add ea_inode support to set
xattr") broke the fix that was addressed in commit 0ee1eaf70c25
("libext2fs: translate internal ext4 acl to Posix ACL in
ext2fs_xattr_[sg]et()"). This was because although we calculated what
the correct on-disk ACL representation would be, we didn't actually
*store* it, but instead stored the passed-in Posix ACL memory
representation instead.
...because there could be superblock updates in the journal that haven't
yet been checkpointed. The online scan is perfectly capable of tallying
the free blocks on its own, so teach it do that and make a more accurate
report.
Reported-by: Elana Hashman <Elana.Hashman@twosigma.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 24 Nov 2018 03:34:31 +0000 (22:34 -0500)]
tune2fs: fix false warning that a UUID change will take a long time
If the file system only has the flex_bg feature enabled (with out the
metadata_csum feature enabled), it won't take a long time time fix up
the checksums after changing the UUID. While it does need to
recalculate all of the checksums in the block group descriptors, that
doesn't take a long time.
Also, if the ea_data feature is enabled, changing the UUID will also
take a long time, and we weren't warning the user about that case.
Fix up the warning message so it doesn't mislead people, and is more
accurate.
Theodore Ts'o [Thu, 22 Nov 2018 23:01:56 +0000 (18:01 -0500)]
mk_cmds: don't use explicit pathname for sed
$AWK doesn't use an explicit pathname, and it's perfectly fine to
assume that awk and sed are in the user's PATH. The problem with
using an explicit pathname is that Debian currently allows merged and
non-merged /usr. Avoid using an explicit pathname to prevent
potential problems.
Li Dongyang [Tue, 13 Nov 2018 04:46:03 +0000 (15:46 +1100)]
e2fsck: check xattr 'system.data' before setting inline_data feature
ext2fs_inline_data_size will happy return 0 and set size to
EXT4_MIN_INLINE_DATA_SIZE even when inode doesn't have
xattr 'system.data', a corrupted i_flags could make e2fsck
enable the inline_data on the superblock.
We should only offer to enable inline_data when i_flags is set
and xattr 'system.data' can be found.
Also use correct prompt for PR_1_INLINE_DATA_FEATURE.
Theodore Ts'o [Sun, 21 Oct 2018 11:35:53 +0000 (07:35 -0400)]
e4defrag: handle failure to open the file system gracefully
If e4defrag is run by root, it will try to open the underlying file
system for files that it is trying to defrag so it can get the file
system parameters. It's currently doing this by searching /etc/mtab.
This isn't the best way to go about doing things, but we'll leave it
for now, at least for a maintenance release. (The better way to do
things would be to look up the device using the blkid library, but
that's a more involved change.)
Since the file system parameters isn't strictly speaking necessary
(after all we get by without them when not running as root), we'll
allow e4defrag to continue running if we can't find the file system.
This can happen if /etc/mtab is pointing at /proc/mounts, and the
kernel can't properly identify the root file system, it is reported as
"/dev/root".
Theodore Ts'o [Sat, 20 Oct 2018 13:14:48 +0000 (09:14 -0400)]
resize2fs: update checksums in the extent tree's relocated block
When shrinking an file system, and we need to relocate an inode, the
checksums in its extent tree must get updated to reflect its new inode
number. When doing this, we need to do this *after* we update the
extent tree to reflect any blocks which need to be relocated due to
the file system shrink operation.
Otherwise, in the case where only an interior node of the extent tree
needs to get relocated, and none of the entries in that node need to
be adjusted, the checksum for that interior node is updated in the old
copy of that block, and then after the extent tree is updated to use
the new copy of that interior node, the extent tree is left with an
invalid checksum.
This is a relatively rare case, since it requires the following
conditions to be true:
*) The metadata checksum feature must be enabled.
*) An inode needs to be relocated.
*) The inode needs to have an interior node.
*) The block for that interior node needs to be relocated.
*) None of blocks addressed by entries in that interior node needs
to be relocated.
When all of these conditions are true, though, the file system is left
with corrupted with bad checksum for the extent tree block.
Nicholas Clark [Tue, 16 Oct 2018 19:34:20 +0000 (15:34 -0400)]
fuse2fs: fix fsname option in some cases
FUSE's parser allows command-line options to be specified before
or after the device/image and mount-path. This commit changes the
value of the fsname mount option to be correct even if options are
specified before the target device/image.
Signed-off-by: Nicholas Clark <nicholas.clark@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Nick Kralevich [Thu, 11 Oct 2018 03:35:17 +0000 (20:35 -0700)]
AOSP: android/perms.c: clean up error handling
There are a number of error conditions which, due to the way
ext2fs_dir_iterate2 operates, would not be propagated to the upper
layers of the call stack. As a result, certain error conditions,
such as not having enough room to allocate blocks for SELinux
labels, would fail silently, instead of causing a compile
failure.
As suggested in
https://android-review.googlesource.com/c/platform/external/e2fsprogs/+/324363
, add a error field to the caller's private data structure, and use the
bit in the field to indicate an error condition. Now, certain errors
which were silently ignored will cause a compile failure when compiling
Android.
Test: Artifically modify selabel_lookup() to return a failure, and
verify Android doesn't compile.
Test: Verify Android compiles under normal circumstances.
Test: Artifically modify ino_add_xattr() to return a failure, and
verify Android doesn't compile.
Bug: 117502873
Bug: 117567573
Bug: 117473440
Lukas Czerner [Tue, 14 Aug 2018 14:37:53 +0000 (16:37 +0200)]
libe2p: avoid segfault when s_nr_users is too high
Currently in e2fsprogs tools it's possible to access out of bounds
memory when reading list of ids sharing a journal log
(journal_superblock_t->s_users[]) in case where s_nr_users is too high.
This is because we never check whether the s_nr_users fits into the
restriction of JFS_USERS_MAX. Fix it by checking that nr_users is not
bigger than JFS_USERS_MAX and error out when possiblem.
Also add test for dumpe2fs. The rest would require involving external
journal which is not possible to test with e2fsprogs test suite at the
moment.
Theodore Ts'o [Fri, 24 Aug 2018 21:12:20 +0000 (17:12 -0400)]
For --enable-subset, change "make install" so it implies "make install-libs"
The e2fsprogs-libs-1.44.x.tar.gz subset distribution had a hack so
that "make install" would install the libraries via an implied "make
install-libs" --- since after all the tarball had was just the
libraries.
This commit makes "make install" behave the same was as the
e2fsprogs-libs distribution in the case of "configure --enable-subset"
Theodore Ts'o [Wed, 22 Aug 2018 20:33:10 +0000 (16:33 -0400)]
debian: remove hard-coded dependency for libss2 in e2fsprogs.shlibs.local
The hard-coded override is very old, and isn't necessary --- in fact,
it's actively harmful because we are using one or two symbols that
were added to libss2 after e2fsprogs 1.34. So remove it.
Eric Biggers [Tue, 21 Aug 2018 17:59:37 +0000 (10:59 -0700)]
e2fsck: allow verity files to have initialized blocks past i_size
Since ext4 verity is going to be an RO_COMPAT feature rather than an
INCOMPAT one, the on-disk i_size of verity inodes needs to be the data
size rather than the full size. Consequently, verity inodes will have
initialized blocks past i_size, containing the Merkle tree and other
verity metadata. So e2fsck must not fix the i_size of such inodes as it
normally would.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Tue, 21 Aug 2018 00:15:52 +0000 (17:15 -0700)]
libext2fs: add verity flag to EXT2_LIB_FEATURE_RO_COMPAT_SUPP
The new ro_compat filesystem feature flag for fs-verity was added to
EXT2_FEATURE_RO_COMPAT_SUPP, but that's not actually used by e2fsprogs
itself. So contrary to the v1.44.4 release notes, 'mke2fs -O verity'
doesn't actually work, nor does e2fsck allow the filesystem to have the
verity feature. Fix it by adding the flag to the correct place
(EXT2_LIB_FEATURE_RO_COMPAT_SUPP) too.
Fixes: faae7aa00df0 ("Reserve codepoints for the fsverity feature.") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Upendra [Thu, 1 Mar 2018 08:15:48 +0000 (17:15 +0900)]
AOSP: blkid: Correct the label name for exfat
Volume label name is 16 bit unicode string according to spec.
Currently blkid labels the device without converting it to
utf-8 chars due to which incorrect label is displayed.
Theodore Ts'o [Sat, 18 Aug 2018 15:19:25 +0000 (11:19 -0400)]
Drop subset tarball and replace it with "configure --enable-subset"
There's no point creating two separate tarfiles. It's not clear
anyone was actually using the subset tarball --- and if they are, they
can replace it by the full source distribution and using the
--enable-subset option.
Theodore Ts'o [Sat, 18 Aug 2018 13:55:26 +0000 (09:55 -0400)]
Remove obsolete files from the repository
They are removed when we generate the tarball, and they serve no
useful purpose, so let's remove them. This will be help us to create
the tarball using git archive more easily.
Theodore Ts'o [Sat, 18 Aug 2018 04:06:40 +0000 (00:06 -0400)]
gen-tarball: handle symlinks correctly in the created the tar file
Now that we use a symlink for the top-level RELEASE-NOTES file, the
previous strategy of using a symlink to force the prefix in the tar
file and then using tar -h won't work. So change how we generate the
tar file to take advantage of GNU tar's --transform option.
Lukas Czerner [Wed, 8 Aug 2018 11:52:56 +0000 (13:52 +0200)]
e2fsck: remove resize inode if both resize_inode and meta_bg are enabled
Previous e2fsprogs versions allowed to create a file system with both
resize_inode and meta_bg enabled. This was fixed by upstream commit 42e77d5d ("libext2fs: don't create filesystems with meta_bg and resize_inode")
However e2fsck still does not recognize the conflict and will attempt to
clear and recreate resize_inode if it's corrupted due to this incompatible
feature combination, though it will create it in the same wrong layout.
Fix it by teaching e2fsck to recognize resize_inode and meta_bg
conflict and fixing it by disabling and clearing resize inode.
David Anderson [Thu, 12 Jul 2018 23:55:08 +0000 (16:55 -0700)]
AOSP: Update Android.bp files.
debugfs now links to a new libext2_support library, which only includes
cstring.c (Android separates other pieces of libsupport into separate
libraries).
Jaco Kroon [Thu, 2 Aug 2018 18:06:46 +0000 (20:06 +0200)]
debugfs: fix ncheck so it handles hard links correctly
Due to hard links inodes can have multiple names (except for folders),
ncheck should find all of the names (equal to the number of links to the
inodes, directories excepted), not names to the count of the provided
inodes.
Theodore Ts'o [Sun, 5 Aug 2018 01:06:19 +0000 (21:06 -0400)]
debian: stop using symlinks to save space on *-dev packages
Using symlinks to save space on duplicate copies of the
/usr/share/doc/*/changelog.Debian.gz is a bit of a mess, since when
the foo-dev package is removed, the files in /usr/share/doc/libfoo/*
get removed, which means the copyright file gets removed.
So stop doing this altogether, and set up maintainer scripts to clean
up the mess so that the symlinks are removed when the packages get
upgraded.
resize2fs: Remove the real kilobytes rant from man page
Remove the rant about the "real" kilobytes from the man page and just
make it more clear that the suffixed units are representing power-of-two
units as we do in mke2fs man page. Also add terabytes to the list.
Li Dongyang [Sat, 14 Jul 2018 00:42:48 +0000 (18:42 -0600)]
Fix compile error and warnings for old gcc versions
-Wimplicit-fallthrough option was added in gcc7 and
-Wpedantic was added in gcc4.8, while #pragma GCC diagnostic
support was not available until gcc4.6
We got following warnings:
../lib/ext2fs/fiemap.h:35: warning: expected [error|warning|ignored] after ‘#pragma GCC diagnostic’
../lib/ext2fs/fiemap.h:36: warning: unknown option after ‘#pragma GCC diagnostic’ kind
../lib/ext2fs/fiemap.h:38: warning: expected [error|warning|ignored] after ‘#pragma GCC diagnostic’
and error:
filefrag.c: In function ‘main’:
filefrag.c:577: error: #pragma GCC diagnostic not allowed inside functions
filefrag.c:578: error: #pragma GCC diagnostic not allowed inside functions
filefrag.c:595: error: #pragma GCC diagnostic not allowed inside functions
when compiling latest e2fsprogs with a gcc older than 4.6
e.g. on CentOS 6.9
Signed-off-by: Li Dongyang <dongyangli@ddn.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: call fsync(2) to clear stale errors for a new a unix I/O channel
Newer versions of Linux will retain errors and allow them to be
returned by fsync() or close(), even if the error happened before the
file descriptor was opened. This was to accomodate Postgres's
"interesting" method of error collection.
Unfortunately, for e2fsck, we never want to see stale errors, as this
can cause xfstests (such generic/347) to fail with a false positive.
e2fsck: fix kernel compat functions to use kernel error return conventions
Fix journal_bmap() and sync_blockdev() to use the kernel error
convetions (e.g., -EIO instead of EIO) since they are called by
reovery.c, which is shared userspace / kernel code.
Without this, e2fsck might print an error message like this:
/sbin/e2fsck: Unknown code ____ 251 while recovering journal of /dev/mapper/thin-vol
instead of what it should have printed which was this:
/sbin/e2fsck: Input/output error while recovering journal of /dev/mapper/thin-vol
Andreas Dilger [Mon, 2 Jul 2018 21:06:32 +0000 (17:06 -0400)]
tests: make generated test scripts read-only
Make generated test scripts read-only, to avoid errors by developers
editing the generated test scripts and then having them accidentally
clobbered when "make" is run again.
Change-Id: I60d417b816b7c559b5e05baf4167fc2cf2a871cf Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 29 Jun 2018 15:35:58 +0000 (11:35 -0400)]
e2fsck: if user declines to fix s_inodes_acount, abort
A recent commit, d74eb7ef: "e2fsck: handle s_inodes_count corruption
properly" tries to fix s_inodes_count. However, if the user declines
to fix the invalid s_inodes_count, e2fsck needs to abort since
otherwise we can crash due to the insane s_inodes_count.
Theodore Ts'o [Thu, 28 Jun 2018 15:10:58 +0000 (11:10 -0400)]
e2fsck: fix hang in readahead code if last block in fs is metadata
If the last block the file system is metadata (very rare, but it can
happen, especially if e2fsck needs to relocate a bitmap block), e2fsck
will hang while trying to do readahead.
Theodore Ts'o [Tue, 26 Jun 2018 19:21:28 +0000 (15:21 -0400)]
tests, mke2fs: add option to suppress xattr copying to fix f_extent_htree
If the developer is running with SELinux enabled on /tmp, the
f_extent_htree regression test will fail because mke2fs by default
copies the extended attributes into the newly created file system (if
a directory hierarchy is specified via the -d option).
Fix this by adding a new extended option to mke2fs, -E no_copy_xattrs
and using it in f_extent_htree's test script.
Theodore Ts'o [Tue, 26 Jun 2018 13:59:19 +0000 (09:59 -0400)]
tests: explicitly force resize2fs's use of lazy (or not)
Resize2fs will not enable lazy_itable if the kernel apparently does
not support that feature. This will cause spurious test failures when
the tests are running on such a system (or where sysfs is not
mounted).
So for the purposes of the regression test we need to force the use of
lazy_itable so that the results conform to expected golden output.
Theodore Ts'o [Tue, 26 Jun 2018 13:02:46 +0000 (09:02 -0400)]
debugfs: fix mknod command on some 32-bit platforms due to LFS
On some platforms the layout of struct stat changes if
_FILE_OFFSET_BITS is set to 64 (which force the use of 64-bit types
and 64-bit variants for system calls such as stat, lseek, etc.)
This is not true (mercifully) on i386, but it is true for the 32-bit
mips platform on Linux. This caused debugfs's mknod command to fail,
since it used struct stat to pass the desired st_mode and st_rdev
fields for the to-be-created device file or FIFO, and this would be
different for create_inode.c and debugfs.c.
Linking together different object files together compiled with
different values of _FILE_OFFSET_BITS is perilous, but for now, let's
fix the specific problem by passing the two fields in the stat
structure that we really care about. This fixes two regression tests
on the 32-bit MIPS platform: d_special_files and r_move_itable.
Theodore Ts'o [Mon, 25 Jun 2018 01:09:59 +0000 (21:09 -0400)]
e2fsck: only retry to open the file system when the superblock is corrupt
Previously we were always retrying the ext2fs_open2() call with the
EXT2_FLAG_IGNORE_SB_ERRORS flag. It only makes sense to do this if
the superblock is reported as corrupt. Otherwise, it's a waste of
time, and results in printing an extra set of error messages to annoy
the user.