Theodore Ts'o [Mon, 25 Jun 2018 01:09:59 +0000 (21:09 -0400)]
e2fsck: only retry to open the file system when the superblock is corrupt
Previously we were always retrying the ext2fs_open2() call with the
EXT2_FLAG_IGNORE_SB_ERRORS flag. It only makes sense to do this if
the superblock is reported as corrupt. Otherwise, it's a waste of
time, and results in printing an extra set of error messages to annoy
the user.
Theodore Ts'o [Sun, 24 Jun 2018 22:30:50 +0000 (18:30 -0400)]
tests: fix i_qcow test to properly test e2image's qcow-to-raw functionality
The test script was not properly checking e2image's ability to convert
a qcow2 image back to a raw image. Fix this, and also change it to
use /tmp for the temporary files to speed things up systems with a
ramdisk for /tmp.
Theodore Ts'o [Sun, 24 Jun 2018 04:32:51 +0000 (00:32 -0400)]
e2image: calculate progress rate using floating point
One of the divisions used when caluclate the progress rate was being
done as an integer instead of floating-point operation. This could
cause the end result to not be as accurate as we would like.
This change ensures that the root inode has the correct uid/gid by
calling androidify_inode() on it. It also sets android_configure = 1
when the u/gid mapping is provided.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Google-Bug-Id: 78561186
Google-Bug-Id: 110415537
Test: $ m # With BOARD_SYSTEMIMAGE_UID_MAP=655360 0 5000
$ simg2img ${OUT}/{system.img,system.raw.img}
$ mkdir tmp
$ sudo mount -o loop ${OUT}/system.raw.img tmp
$ stat --format="%u %g" tmp
655360 655360
Change-Id: I842a32b6d64439f9f16b9c6b66de20d4e4418aa8
From AOSP commit: ee3f0f37c34d68aa7a96a36dea369acae312772c
Eric Sandeen [Sat, 23 Jun 2018 22:04:19 +0000 (18:04 -0400)]
libext2fs: remove c99 idiom to fix build
hashmap.c: In function ‘ext2fs_hashmap_free’:
hashmap.c:72:2: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (size_t i = 0; i < h->size; ++i) {
^
hashmap.c:72:2: note: use option -std=c99 or -std=gnu99 to compile your code
make[2]: *** [hashmap.o] Error 1
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 23 Jun 2018 20:26:41 +0000 (16:26 -0400)]
tests: PAGE_SIZE larger than blocksize with hole
Verify correct operation in the case of writing files with allocated
blocks at the end of the file beyond i_size. This can happen for
PAGE_SIZE > blocksize, or through fallocate().
The f_eofblocks thest checks quite the same situation, but for broken
file system. This test is about correct file.
Theodore Ts'o [Sat, 23 Jun 2018 03:05:49 +0000 (23:05 -0400)]
libext2fs: simplify extent_fallocate()
The goal_distance variable is used in only one place in the
extent_fallocate() function, so set it right before it is used, and
eliminate one assignment of goal_distance which is never used before
it gets overwritten.
Theodore Ts'o [Sat, 23 Jun 2018 02:53:00 +0000 (22:53 -0400)]
libsupport: fix (very potential) NULL dereference
It's a very rare set of circumstances (requiring deleting profile
sections, which e2fsprogs doesn't do) that might trigger a NULL
derference, but fix it for robustness's sake, maybe some day someone
will use that functionality.
Theodore Ts'o [Sat, 23 Jun 2018 02:33:39 +0000 (22:33 -0400)]
libext2fs: fix uninitialized variable dereference in extent_bmap()
The code path was failing to initialize the logial block number hint
in the allocation context. It's not currently being used, but it
might be in the future, and it trips warnings in static code
analyzers.
Shuichi Ihara [Wed, 2 May 2018 04:26:06 +0000 (22:26 -0600)]
misc: add e2mmpstatus utility via dumpe2fs
e2mmpstatus is a Multi-Mount Protection (MMP) helper utility to read
an MMP block to see if it is being updated. It can also output the
latest update time, nodename, and device from the MMP block.
This is useful for HA and other maintenance scripts to determine if
the filesystem is in use on another node, and which node it is.
Signed-off-by: Shuichi Ihara <sihara@ddn.com> Signed-off-by: Li Xi <lixi@ddn.com> Signed-off-by: Wang Shilong <wshilong@ddn.com>
Moved e2mmpstatus checking/dumping code to be part of dumpe2fs rather
than a standalone program, using the "-m" option to check MMP status,
and "-i" to dump info. If dumpe2fs is called as "e2mmpstatus" (and
also "mmpstatus" for compatibility reasons), assume "-m" is specified.
Re-use the existing MMP block handing routines (with some changes) to
check and dump the MMP block, rather than adding duplicate versions.
Modify dumpe2fs to exit with a non-zero error code if there is an
error while reading the filesystem metadata or MMP block, or if
"-m" is used with the "mmp" feature and is in use by another node.
Add a configure check for gethostname() rather than depending on
_BSD_SOURCE or _XOPEN_SOURCE to be set.
Update the f_mmp, m_mmp, m_mmp_bad_csum, and m_mmp_bad_magic tests
to use e2mmpstatus to check and dump the MMP state before and after
e2fsck is run to verify that the tool is working correctly.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 22 Jun 2018 22:08:54 +0000 (18:08 -0400)]
e2fsck: set dir_nlink feature if large dir exists
If there is a directory with more than EXT2_LINK_MAX (65000)
subdirectories, but the DIR_NLINK feature is not set in the
superblock, the feature should be set before continuing on
to change the on-disk directory link count to 1.
While most filesystems should have DIR_NLINK set (it was set
by default for all ext4 filesystems, and all kernels between
2.6.23 and 4.12 automatically set it if the directory link
count grew too large), it is possible that this flag is lost
due to disk corruption or for an upgraded filesystem. We no
longer want kernels to automatically enable features.
Addresses: https://bugzilla.kernel.org/show_bug.cgi?id=196405 Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 22 Jun 2018 21:54:50 +0000 (17:54 -0400)]
tests: remove pre-generated f_extent_htree image
The f_extent_htree test depends on the "mke2fs -d" functionality
that did not exist in the maint branch when the patch was landed.
On master this functionality exists and should be used instead of
a pre-generated image. That exercises the "mke2fs -d" functionality
as well as e2fsck better.
[ Fixed to add missing expect.pre.? files --tytso ]
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Wed, 2 May 2018 04:26:03 +0000 (22:26 -0600)]
tests: don't run sed multiple times on test output
Don't call sed multiple times on the output, and avoid the use
of temporary files, or if possible. It would be convenient to
use "sed -i" to only update the output file once, but this is
not portable to all platforms.
[ Fixed a few test regression failures --tytso ]
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Wed, 2 May 2018 04:26:02 +0000 (22:26 -0600)]
tests: remove redundant sed filtering
Now that the majority of device name filtering is in filter.sed,
it does not need to be specified explicitly for every test.
Fix the error message printed in debugfs when opening the device
to match that used in other tools. This simplifies the filtering,
and will be helpful if debugfs messages are internationalized.
[ Fixed up some test failures in m_hugefile t_uninit_bg_rm --tytso ]
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 22 Jun 2018 15:43:54 +0000 (11:43 -0400)]
tests: clean up $DEBUGFS_EXE usage in scripts
Instead of putting the entire test script under an implicit
"if test -x $DEBUGFS_EXE" conditional (sometimes indenting
the code, and sometimes not), rather check for the reverse
and exit the test script early if $DEBUGFS_EXE is missing.
In some places, tests were running $DEBUGFS_EXE directly,
when they should be running $DEBUGFS (which will run with
Fortify, or other options).
[ Fixed up missing exit statement in f_detect_junk. --tytso ]
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 22 Jun 2018 15:42:14 +0000 (11:42 -0400)]
tests: don't use a sparse test file
If the TEST_BITS file is sparse, then the "debugfs -R write"
command may skip holes in the file when copying it into the
test image (depending on whether SEEK_HOLE/SEEK_DATA and/or
FIEMAP are available in the copy_file() function).
This was causing test failures on MacOS in the f_dup_resize
and d_loaddump tests because the TEST_BITS file was the
compiled "debugfs" binary, which apparently has holes when
built on MacOS, and the number of blocks allocated in the
test image was reduced as a result. This caused the expect
output to differ in the summary line and resulted in failure.
Instead of using the debugfs binary for TEST_BITS, generate
a temporary file using /dev/urandom, if available. If not,
fall back to the old behaviour or using debugfs.
[ Fixed up use of non-POSIX /bin/sh constructs and add test_data.tmp to
.gitignore and files to be deleted on a make clean. --tytso ]
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 22 Jun 2018 14:46:52 +0000 (10:46 -0400)]
libext2fs: don't corrupt an blkmap64_rb when marking a range of size zero
Calling ext2fs_mark_block_bitmap_range2() with a count of zero can end
up corrupting the red-black block bitmap structure, since a an entry
in the rbtree with zero-length extent can end up causing the
find_first_{zero,set} operations to return incorrect results.
This was found by Adam Buchbinder, who created a fuzzed file system
using which AFL that caused e2fsck to hang in an infinite loop in in
e2fsck's readahead code.
Jiyong Park [Thu, 24 May 2018 05:07:52 +0000 (14:07 +0900)]
AOSP: Mark as recovery_available:true
Libraries that are direct or indirect dependencies of modules installed
to recovery partition (e.g. adbd) are marked as recovery_available:
true. This allows a recovery variant of the lib is created when it is
depended by other recovery or recovery_available modules.
David Anderson [Fri, 11 May 2018 20:34:34 +0000 (13:34 -0700)]
AOSP: e2fsck: imply -f when using -E unshare_blocks
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Google-Bug-Id: 64109868
Test: e2fsck -E unshare_blocks does a full scan
Change-Id: Idc36ceba3bf24e1fb1487feedefe9a68f9acc7f3
From AOSP commit: 7c180d6598363722de6195d142d7677bbc2b0161
David Anderson [Fri, 27 Apr 2018 23:10:29 +0000 (16:10 -0700)]
AOSP: e2fsck: allow read-only testing if -E unshare_blocks will succeed
If -E unshare_blocks is used with -n, it will normally fail since the
filesystem is read-only. For Android's "adb remount" it is more useful
to report whether or not the unshare operation would succeed, were the
filesystem writable. We do that here by ignoring certain write
operations if -E unshare_blocks is specified with -n. It is not perfect,
since the actual unshare operation could still fail (for example if
new extents need to consume additional blocks).
Tao Bao [Mon, 23 Apr 2018 21:10:59 +0000 (14:10 -0700)]
AOSP: libext2fs: Track the libsparse API change.
libsparse is updating the 'len' parameter, from 'int' to 'size_t', in
the callback parameter of sparse_file_foreach_chunk(). The value
represents the chunk size, which could be legitimately larger than
INT_MAX.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Google-Bug-Id: 78432315
Test: `m dist` with aosp_marlin-userdebug
Change-Id: I35f8a528aff461ce4d3b492a7ca2d4a23592be54
From AOSP commit: 019bed178935585d28cf702a8090a5598415312b
David Anderson [Mon, 5 Mar 2018 20:52:13 +0000 (12:52 -0800)]
AOSP: e2fsck: Add an extended option for unsharing blocks.
Add an -E unshare_blocks flag for unsharing blocks that were created for
a filesystem with block sharing enabled. If the filesystem does not have
this feature enabled, the flag has no effect. If the filesystem does not
have free space, e2fsck will error.
Jan Kara [Tue, 19 Jun 2018 15:34:46 +0000 (11:34 -0400)]
e2fsck: report only one sb corruption
check_super_value() does not terminate in case of error anymore since c8b20b40ebf0 "misc: add plausibility checks to
debugfs/tune2fs/dumpe2fs/e2fsck" which removed the PR_FATAL flag from
PR_0_SB_CORRUPT problem. This results in potentially many errors for
superblock being printed including the long message about how to deal
with corrupted superblock. Restore the original behavior of reporting
only one error and also remove the comments 'never get here' as they are
not true anymore.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 15:15:53 +0000 (11:15 -0400)]
debugfs: allow read-write opening in catastrophic mode
Allow filesystem to be open read-write in catastrophic mode so that one
can fixup e.g. superblock breakage. The CHECK_FS_BITMAPS flag to
common_args_process() still guards us from doing operations on bitmaps
which we don't load in this mode.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 15:23:37 +0000 (11:23 -0400)]
e2fsck: handle s_inodes_count corruption properly
When s_inodes_count would overflow given number of groups and inodes per
group, we cannot currently fix the breakage in e2fsck as that requires
trimming number of groups or inodes per group which both means data &
inode migration etc. Just trimming sb->s_inodes_count is not enough as
kernel's inode allocation code is not able to handle filesystems where
not all inodes in the last group are usable. So don't pretend we can fix
s_inodes_count overflow by just trimming the s_inodes_count value.
When s_inodes_count is just wrong but will not overflow, let's fix it.
Also move this check before we use s_inodes_count for checking
s_first_ino.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 12 Jun 2018 09:53:23 +0000 (11:53 +0200)]
e2fsck: allow to fix some superblock corruptions
Add a flag to ext2fs_open() which allows to open a filesystem even if
superblock is somewhat inconsistent. Use this flag from e2fsck as a last
resort to try to fix the superblok.
Currently, the flag does nothing. We'll relax checks in ext2fs_open()
once e2fsck is able to handle corresponding corruption gracefully.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 14:45:16 +0000 (10:45 -0400)]
libext2fs: don't create filesystems with meta_bg and resize_inode
ext2fs_initialize() may end up enabling meta_bg feature for filesystem
which have resize_inode. Such combination is invalid to make sure we
disable resize_inode when enabling meta_bg.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Tue, 19 Jun 2018 03:19:27 +0000 (23:19 -0400)]
libext2fs: fix possible inode count overflow when creating fs
If blocks count is exactly 1<<32, then the code computing number of
inode count in ext2fs_initialize() will overflow and set number of
inodes to 0 (which will be later fixed up to EXT2_FIRST_INODE(super)+1).
Fix the off-by-one bug in the check.
Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 13 Jun 2018 22:55:56 +0000 (18:55 -0400)]
libext2fs: allow file systems which have insane values in s_desc_size
If the 64-bit feature is not set, the kernel does not pay attention to
the s_desc_size field in the superblock. Change ext2fs_open2() and
other library functions to similarly ignore s_desc_size if the 64-bit
feature is not set.
The EXT2_DESC_SIZE macro should be used in most cases instead of
referecing the s_desc_size field directly.
Theodore Ts'o [Sun, 27 May 2018 03:11:18 +0000 (23:11 -0400)]
debugfs: add -b and -e options to the inode_dump command
Teach the inode_dump command to dump out just the i_block array and
the extra space in the inode, as a convenience to someone
investigating a corrupted inode.
Chandan Rajendra [Wed, 23 May 2018 03:22:07 +0000 (08:52 +0530)]
e2fsprogs: Use 32-bit variant of ext2fs_swab to read i_projid
i_projid is a 32-bit field of the inode. Hence this commit uses
ext2fs_swab32() to convert the i_projid field from the on-disk little
endian format to the host cpu format. Without this change, project quota
consistency check used to fail on big endian powerpc systems.
Theodore Ts'o [Mon, 14 May 2018 22:22:09 +0000 (18:22 -0400)]
mke2fs: print error string if ext2fs_close_free() fails
There are multiple reasons why ext2fs_close_free() might fail, not
just an I/O error while writing out a backup superblock. Print the
error code, and then allow mke2fs to exit with an exit status code of
1, instead of whatever random error code ext2fs_close_free() might
have returned with.
This is mostly automatic replace of Free Software Foundation address in
all our files with the correct address that can be found at
http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
e2image: fix metadata image handling on big endian systems
Currently e2image metadata image handling and creating is completely
broken on big endian systems. It just does not care about endianness at
all. This was uncovered With addition of i_bitmaps test, which is the
first test that actually tests e2image metadata image.
Fix it by making sure that all on-disk metadata that we write and read
to/from the metadata image is properly converted.
Make sure we can handle a maliciously created file system containing
an inode containing an extended attribute whose e_value_inum points
back at itself.
An inode containing the value for an extended attribute (aka an
ea_in_inode) must not have the INLINE_DATA flag and must have the
EA_INODE flag set. Enforcing this prevents e2fsck and debugfs crashes
caused by a maliciously crafted file system containing an inode which
has both the EA_INODE and INLINE_DATA flags set, and where that inode
has an extended attribute whose e_value_inum points to itself.
Theodore Ts'o [Sat, 31 Mar 2018 20:38:48 +0000 (16:38 -0400)]
tests: use mke2fs and debugfs from the build tree
The tests f_bigalloc_badinode and f_bigalloc_orphan_list were not
using the version of mke2fs and debugfs from the build tree, and if
mke2fs and debugfs are not in the user's PATH, these tests would fail.
Fix this.
Darrick J. Wong [Sun, 25 Mar 2018 19:01:33 +0000 (12:01 -0700)]
e2fsck: adjust quota counters when clearing orphaned inodes
If e2fsck encounters a filesystem that supports internal quotas, it is
responsible for adjusting the quota counters if it decides to clear any
orphaned inodes. Therefore, we must read the quota files, adjust the
counters, and write the quota files back out when we are done.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Thu, 29 Mar 2018 18:36:54 +0000 (12:36 -0600)]
filefrag: avoid temporary buffer overflow
If an unknown flag is present in a FIEMAP extent, it is printed as a
hex value into a temporary buffer before adding it to the flags. If
that unknown flag is over 0xfff then it will overflow the temporary
buffer.
Reported-by: Sarah Liu <wei3.liu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-10335 Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 24 Mar 2018 15:39:09 +0000 (11:39 -0400)]
tests: explicitly specify 1k block sizes when creating test file systems
On the Hurd, mke2fs will force the use of 4k block sizes by default
because the Hurd's implemntation of ext2 doesn't support any other
block sizes. This causes spurious test failures. Since these test
are testing e2fsprogs functionality, force the use of 1k block sizes
so the test output matches the expected output.
projects / thirdparty / e2fsprogs.git / log
