David Anderson [Fri, 11 May 2018 20:34:34 +0000 (13:34 -0700)]
AOSP: e2fsck: imply -f when using -E unshare_blocks
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Google-Bug-Id: 64109868
Test: e2fsck -E unshare_blocks does a full scan
Change-Id: Idc36ceba3bf24e1fb1487feedefe9a68f9acc7f3
From AOSP commit: 7c180d6598363722de6195d142d7677bbc2b0161
David Anderson [Fri, 27 Apr 2018 23:10:29 +0000 (16:10 -0700)]
AOSP: e2fsck: allow read-only testing if -E unshare_blocks will succeed
If -E unshare_blocks is used with -n, it will normally fail since the
filesystem is read-only. For Android's "adb remount" it is more useful
to report whether or not the unshare operation would succeed, were the
filesystem writable. We do that here by ignoring certain write
operations if -E unshare_blocks is specified with -n. It is not perfect,
since the actual unshare operation could still fail (for example if
new extents need to consume additional blocks).
Tao Bao [Mon, 23 Apr 2018 21:10:59 +0000 (14:10 -0700)]
AOSP: libext2fs: Track the libsparse API change.
libsparse is updating the 'len' parameter, from 'int' to 'size_t', in
the callback parameter of sparse_file_foreach_chunk(). The value
represents the chunk size, which could be legitimately larger than
INT_MAX.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Google-Bug-Id: 78432315
Test: `m dist` with aosp_marlin-userdebug
Change-Id: I35f8a528aff461ce4d3b492a7ca2d4a23592be54
From AOSP commit: 019bed178935585d28cf702a8090a5598415312b
David Anderson [Mon, 5 Mar 2018 20:52:13 +0000 (12:52 -0800)]
AOSP: e2fsck: Add an extended option for unsharing blocks.
Add an -E unshare_blocks flag for unsharing blocks that were created for
a filesystem with block sharing enabled. If the filesystem does not have
this feature enabled, the flag has no effect. If the filesystem does not
have free space, e2fsck will error.
Jan Kara [Tue, 19 Jun 2018 15:34:46 +0000 (11:34 -0400)]
e2fsck: report only one sb corruption
check_super_value() does not terminate in case of error anymore since c8b20b40ebf0 "misc: add plausibility checks to
debugfs/tune2fs/dumpe2fs/e2fsck" which removed the PR_FATAL flag from
PR_0_SB_CORRUPT problem. This results in potentially many errors for
superblock being printed including the long message about how to deal
with corrupted superblock. Restore the original behavior of reporting
only one error and also remove the comments 'never get here' as they are
not true anymore.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 15:15:53 +0000 (11:15 -0400)]
debugfs: allow read-write opening in catastrophic mode
Allow filesystem to be open read-write in catastrophic mode so that one
can fixup e.g. superblock breakage. The CHECK_FS_BITMAPS flag to
common_args_process() still guards us from doing operations on bitmaps
which we don't load in this mode.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 15:23:37 +0000 (11:23 -0400)]
e2fsck: handle s_inodes_count corruption properly
When s_inodes_count would overflow given number of groups and inodes per
group, we cannot currently fix the breakage in e2fsck as that requires
trimming number of groups or inodes per group which both means data &
inode migration etc. Just trimming sb->s_inodes_count is not enough as
kernel's inode allocation code is not able to handle filesystems where
not all inodes in the last group are usable. So don't pretend we can fix
s_inodes_count overflow by just trimming the s_inodes_count value.
When s_inodes_count is just wrong but will not overflow, let's fix it.
Also move this check before we use s_inodes_count for checking
s_first_ino.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 12 Jun 2018 09:53:23 +0000 (11:53 +0200)]
e2fsck: allow to fix some superblock corruptions
Add a flag to ext2fs_open() which allows to open a filesystem even if
superblock is somewhat inconsistent. Use this flag from e2fsck as a last
resort to try to fix the superblok.
Currently, the flag does nothing. We'll relax checks in ext2fs_open()
once e2fsck is able to handle corresponding corruption gracefully.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 14:45:16 +0000 (10:45 -0400)]
libext2fs: don't create filesystems with meta_bg and resize_inode
ext2fs_initialize() may end up enabling meta_bg feature for filesystem
which have resize_inode. Such combination is invalid to make sure we
disable resize_inode when enabling meta_bg.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 03:19:27 +0000 (23:19 -0400)]
libext2fs: fix possible inode count overflow when creating fs
If blocks count is exactly 1<<32, then the code computing number of
inode count in ext2fs_initialize() will overflow and set number of
inodes to 0 (which will be later fixed up to EXT2_FIRST_INODE(super)+1).
Fix the off-by-one bug in the check.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 13 Jun 2018 22:55:56 +0000 (18:55 -0400)]
libext2fs: allow file systems which have insane values in s_desc_size
If the 64-bit feature is not set, the kernel does not pay attention to
the s_desc_size field in the superblock. Change ext2fs_open2() and
other library functions to similarly ignore s_desc_size if the 64-bit
feature is not set.
The EXT2_DESC_SIZE macro should be used in most cases instead of
referecing the s_desc_size field directly.
Theodore Ts'o [Sun, 27 May 2018 03:11:18 +0000 (23:11 -0400)]
debugfs: add -b and -e options to the inode_dump command
Teach the inode_dump command to dump out just the i_block array and
the extra space in the inode, as a convenience to someone
investigating a corrupted inode.
Chandan Rajendra [Wed, 23 May 2018 03:22:07 +0000 (08:52 +0530)]
e2fsprogs: Use 32-bit variant of ext2fs_swab to read i_projid
i_projid is a 32-bit field of the inode. Hence this commit uses
ext2fs_swab32() to convert the i_projid field from the on-disk little
endian format to the host cpu format. Without this change, project quota
consistency check used to fail on big endian powerpc systems.
Theodore Ts'o [Mon, 14 May 2018 22:22:09 +0000 (18:22 -0400)]
mke2fs: print error string if ext2fs_close_free() fails
There are multiple reasons why ext2fs_close_free() might fail, not
just an I/O error while writing out a backup superblock. Print the
error code, and then allow mke2fs to exit with an exit status code of
1, instead of whatever random error code ext2fs_close_free() might
have returned with.
This is mostly automatic replace of Free Software Foundation address in
all our files with the correct address that can be found at
http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
e2image: fix metadata image handling on big endian systems
Currently e2image metadata image handling and creating is completely
broken on big endian systems. It just does not care about endianness at
all. This was uncovered With addition of i_bitmaps test, which is the
first test that actually tests e2image metadata image.
Fix it by making sure that all on-disk metadata that we write and read
to/from the metadata image is properly converted.
Make sure we can handle a maliciously created file system containing
an inode containing an extended attribute whose e_value_inum points
back at itself.
An inode containing the value for an extended attribute (aka an
ea_in_inode) must not have the INLINE_DATA flag and must have the
EA_INODE flag set. Enforcing this prevents e2fsck and debugfs crashes
caused by a maliciously crafted file system containing an inode which
has both the EA_INODE and INLINE_DATA flags set, and where that inode
has an extended attribute whose e_value_inum points to itself.
Theodore Ts'o [Sat, 31 Mar 2018 20:38:48 +0000 (16:38 -0400)]
tests: use mke2fs and debugfs from the build tree
The tests f_bigalloc_badinode and f_bigalloc_orphan_list were not
using the version of mke2fs and debugfs from the build tree, and if
mke2fs and debugfs are not in the user's PATH, these tests would fail.
Fix this.
Darrick J. Wong [Sun, 25 Mar 2018 19:01:33 +0000 (12:01 -0700)]
e2fsck: adjust quota counters when clearing orphaned inodes
If e2fsck encounters a filesystem that supports internal quotas, it is
responsible for adjusting the quota counters if it decides to clear any
orphaned inodes. Therefore, we must read the quota files, adjust the
counters, and write the quota files back out when we are done.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Thu, 29 Mar 2018 18:36:54 +0000 (12:36 -0600)]
filefrag: avoid temporary buffer overflow
If an unknown flag is present in a FIEMAP extent, it is printed as a
hex value into a temporary buffer before adding it to the flags. If
that unknown flag is over 0xfff then it will overflow the temporary
buffer.
Reported-by: Sarah Liu <wei3.liu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-10335 Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 24 Mar 2018 15:39:09 +0000 (11:39 -0400)]
tests: explicitly specify 1k block sizes when creating test file systems
On the Hurd, mke2fs will force the use of 4k block sizes by default
because the Hurd's implemntation of ext2 doesn't support any other
block sizes. This causes spurious test failures. Since these test
are testing e2fsprogs functionality, force the use of 1k block sizes
so the test output matches the expected output.
harshads [Fri, 23 Mar 2018 18:34:05 +0000 (11:34 -0700)]
e2fsck: delete bad inode fix for bigalloc
While deleting a bad inode in fsck pass2, we should remove clusters
only once. We do it by remembering last released cluster while
deleting clusters one by one.
harshads [Fri, 23 Mar 2018 18:32:49 +0000 (11:32 -0700)]
debugfs: release clusters only once in release_blocks_proc
While killing file by inode in debugfs (kill_file_by_inode), if
bigalloc feature is turned on, release clusters only once. We do it by
remembering the last released cluster while releasing blocks. We
release a cluster only if it is not already released.
Robert Schiele [Tue, 20 Mar 2018 04:38:48 +0000 (00:38 -0400)]
util: allow subst to build in cross build environemnt
In a cross build environment we don't get definition of
HAVE_SYS_STAT_H from config.h, therefore we need to define it locally
in that case similar to HAVE_SYS_TIME_H.
Fixes: 7fd537401270 ("misc: add missing declarations on maint") Signed-off-by: Robert Schiele <rschiele@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 19 Mar 2018 22:23:04 +0000 (18:23 -0400)]
libext2fs: teach e2image imager to handle bigalloc file systems
Previously, "mke2fs -t ext4 -O bigalloc /tmp/foo.img 4G ;
e2image /tmp/foo.img /tmp/foo.e2i" would result in e2image
dumping core. Fix ext2fs_image_bitmap_write() so it handles
file systems with bigalloc correctly.
Kazuya Mio [Sat, 17 Mar 2018 18:56:15 +0000 (14:56 -0400)]
libext2fs: fix ext2fs_open2() error for meta_bg image file
dumpe2fs/debugfs can examine the image file by using the -i option.
However, if meta_bg feature is enabled, dumpe2fs/debugfs cannot open
the image file.
$ dumpe2fs -i test.img
dumpe2fs: Attempt to read block from filesystem resulted in short read while trying to open test.img
Couldn't find valid filesystem superblock.
In case of specifying an image file, the location of block group descriptors
is the same as the case of default filesystem regardless of meta_bg feature.
So if EXT2_FLAG_IMAGE_FILE flag is set in ext2fs_open2(),
don't use the meta_bg handling.
harshads [Tue, 13 Mar 2018 23:19:07 +0000 (16:19 -0700)]
e2fsck: release clusters only once in release_inode_blocks
While releasing inode blocks, if bigalloc feature is turned on,
release clusters only once. We do it by remembering the last released
cluster while iterating through blocks and releasing a cluster only if
it is not already released.
Lukas Czerner [Fri, 9 Mar 2018 11:28:02 +0000 (12:28 +0100)]
e2fsck: fix endianness problem when reading htree nodes
Wrong directory block number can be saved in ->previous on big endian
system in parse_int_node(). Fix it by moving the mask out of the endian
conversion.
Eric Biggers [Sat, 3 Mar 2018 00:59:23 +0000 (16:59 -0800)]
tests: test e2fsck's handling of bad symlinks
Add a test which verifies that e2fsck will detect a variety of bad
symlinks, both fast and slow, and with several combinations of
filesystem features including extents, encryption, and inline_data.
There was already a similar test (f_badsymlinks), but it's an old test
that doesn't use any of these newer filesystem features.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:22 +0000 (16:59 -0800)]
e2fsck: require that fast symlinks don't have EXT4_EXTENTS_FL
It doesn't make sense for EXT4_EXTENTS_FL to be set on a fast symlink.
The kernel doesn't set it, and it ignores it if set. Meanwhile, e2fsck
is stricter: it will try to validate the extent tree, which will almost
certainly fail (assuming the symlink is, in fact, a fast symlink).
Make this behavior more explicit by rejecting EXT4_EXTENTS_FL for fast
symlinks, rather than going ahead and trying to validate an extent tree.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:21 +0000 (16:59 -0800)]
e2fsck: validate the targets of extent-based symlinks
e2fsck is validating the target (requiring that it be NUL-terminated at
i_size, or something a bit different for encrypted symlinks) of slow
symlinks that use a traditional block list but not ones that use an
extent tree. As far as I can tell this is simply a bug: there's no
reason for the representation of the block list to affect how the
symlink target is validated. And either way the kernel won't create
symlinks with embedded NULs and will always add a terminating NUL.
Thus, make e2fsck_pass1_check_symlink() start validating the targets of
extent-based symlinks.
Fixes: 7cadc57780f3 ("e2fsck: Support long symlinks which use extents") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:20 +0000 (16:59 -0800)]
e2fsck: drop redundant checks of symlink i_size
e2fsck_pass1_check_symlink() verifies that the symlink inode's i_size is
less than the buffer length (60 for fast symlinks, fs->blocksize for
slow symlinks). But it also verifies that len == i_size &&
len < buflen, which already implies i_size < buflen. Thus, remove the
redundant checks of i_size.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2fsck validates that unencrypted symlinks have their strlen() equal to
i_size. But it skips the equivalent check of i_size ==
fscrypt_symlink_data.len + 2 for encrypted symlinks. Actually, the
encrypted symlink header is redundant with i_size and shouldn't exist.
But it's there, and the kernel does in fact use the length in the header
instead of i_size -- so e2fsck should validate the header.
Thus, remove the exception for encrypted symlinks, so e2fsck will now
require i_size == fscrypt_symlink_data.len + 2. I think the exception
was only there originally because for encrypted fast symlinks we were
calculating the length using strnlen() which was wrong. But that was
fixed by the previous patch.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:18 +0000 (16:59 -0800)]
e2fsck: validate fscrypt_symlink_data.len for fast symlinks too
Both fast and slow encrypted symlinks are prefixed with the ciphertext
length field (fscrypt_symlink_data.len). But e2fsck was only checking
it for slow symlinks. Start checking it for fast symlinks too. This
matches the kernel handling of encrypted symlinks.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:17 +0000 (16:59 -0800)]
e2fsck: validate that fscrypt_symlink_data.len is not too large
Make e2fsck start validating that the ciphertext length stored in the
header of an encrypted symlink target, plus the header itself, is no
larger than a filesystem block. Previously e2fsck only verified that
this size is not exactly equal to a filesystem block. This was
sufficient for unencrypted symlinks, where the "actual length" is
computed using strnlen(), but not for encrypted symlinks; the kernel
also considers encrypted symlinks with too-large ciphertext length to be
invalid.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:16 +0000 (16:59 -0800)]
e2fsck: fix reading fscrypt_symlink_data.len
The ciphertext length field stored at the beginning of encrypted symlink
targets is 16-bit. But e2fsck_pass1_check_symlink() is reading it as
32-bit. This was apparently left over from an earlier on-disk format
that was not merged. Fix it.
This bug caused a small proportion of encrypted symlinks with 4092-byte
targets to be considered invalid by e2fsck, but otherwise had no effect.
Fixes: 62ad24802c6e ("e2fsck: handle encrypted directories which are indexed using htree") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:15 +0000 (16:59 -0800)]
libext2fs: make sure the system.data xattr gets created
Both the kernel and e2fsck expect that if an inode has inline data, then
it contains a "system.data" xattr -- even if i_size <= 60 so the data
fits entirely in i_block.
But if a symlink of exactly 60 bytes (not counting a NUL terminator) was
created using ext2fs_symlink() and the inline data feature was enabled,
then the symlink inode ended up with inline data but without a
system.data xattr. This is possible because "fast" symlinks store a NUL
terminator but inline data symlinks do not. So 60 bytes is too long for
a real fast symlink, but still short enough to fit the entire target in
i_block as a "slow" symlink using inline data.
Some places use ext2fs_inline_data_init() to ensure the system.data
xattr is created, but the symlink path does not.
Fix this by making ext2fs_inline_data_set() set system.data to an empty
string when i_size <= 60.
Fixes: 54e880b870f7 ("libext2fs: handle inline data in read/write function") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sat, 3 Mar 2018 00:59:14 +0000 (16:59 -0800)]
libext2fs: forbid creating symlinks using entire block
ext2fs_symlink() didn't count the NUL terminator when limiting the
symlink target length to fs->blocksize, so it could create symlinks
using the entire block. Such symlinks are problematic because if
block_size == PAGE_SIZE (as is usually the case), then when following
such a symlink the kernel will truncate the last byte because it needs
to ensure the symlink page is NUL-terminated (see page_get_link()).
Perhaps for that reason, e2fsck considers such symlinks to be invalid
(although only if they use the traditional block list rather than an
extent tree, which seems to be another bug).
Fix this by counting the NUL terminator, thereby decreasing the maximum
accepted symlink target length by 1. Note that this matches the
kernel's behavior in ext4_symlink().
This breaks the test 'f_create_symlinks' which was using debugfs to
create a 1024-byte symlink (not counting the NUL terminator) on a
1024-byte block filesystem. Fix it by removing the leading '/' from the
test's symlink targets so that their lengths are decreased by 1.
Fixes: f01c1a6bce5e ("libext2fs: add the ext2fs_symlink() function") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Fri, 2 Mar 2018 05:21:35 +0000 (21:21 -0800)]
blkid: fix building e2fsprogs with internal libblkid
Building e2fsprogs with the internal libblkid fails for me with the
following compiler error:
In file included from blkidP.h:27:0,
from dev.c:17:
dev.c: In function ‘blkid_free_dev’:
../../lib/blkid/list.h:149:42: error: ‘intptr_t’ undeclared (first use in this function); did you mean ‘__intptr_t’?
((type *)((char *)(ptr)-(unsigned long)(intptr_t)(&((type *)0)->member)))
The problem is that lib/blkid/list.h is now using 'intptr_t', but
stdint.h is not guaranteed to have been included yet. Include it.
[ Changed to use inttypes.h if it is available and fall back to stdint.h
if it is not available - tytso ]
Fixes: 0a3d8041892c ("AOSP: build mke2fs for windows using android mingw library") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 28 Feb 2018 03:15:54 +0000 (22:15 -0500)]
libext2fs: declare local functions static to avoid namespace leakage
The functions, xattr_array_update() and space_used() were accidentally
allowed to be declared extern and were thus made available via the
shared library. Fix this.
Theodore Ts'o [Tue, 27 Feb 2018 22:32:27 +0000 (17:32 -0500)]
Break up RELEASE-NOTES into files in doc/RelNotes
The RELEASE-NOTES file was getting too big as be unweildy. It also
made it hard to have simultaneous releases of older maintenance
releases. So split it up into smaller files in the doc/RelNotes
directory.
Theodore Ts'o [Tue, 27 Feb 2018 06:27:10 +0000 (01:27 -0500)]
resize2fs: remove bigalloc warning for online resizing
The kernel supports online resizing of bigalloc file systems. There's
no real point to have a scary messaging when doing an online resize of
bigalloc file systems, since all of the heavy lifting is done by the
kernel.
projects / thirdparty / e2fsprogs.git / log
