Jan Kara [Mon, 12 Jul 2021 15:43:14 +0000 (17:43 +0200)]
mke2fs: Add orphan_file feature into mke2fs.conf
Enable orphan_file feature by default in larger filesystems. Since the
feature is COMPAT, older kernels will just ignore it and happily work
with the filesystem as well.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Zhiqiang Liu [Wed, 28 Jul 2021 01:56:47 +0000 (09:56 +0800)]
hashmap: change return value type of ext2fs_hashmap_add()
In ext2fs_hashmap_add(), new entry is allocated by calling
malloc(). If malloc() return NULL, it will cause a
segmentation fault problem.
Here, we change return value type of ext2fs_hashmap_add()
from void to int. If allocating new entry fails, we will
return -1, and the callers should also verify the return
value of ext2fs_hashmap_add().
Theodore Ts'o [Tue, 3 Aug 2021 01:08:01 +0000 (21:08 -0400)]
mke2fs: fix creating a file system image w/o a pre-existing file
The mke2fs program should allow creating a file system image when an
explicit file system size is specified, even if the file doesn't yet
exist. By deferring the call to check_plausible() in commit 942b00cb9d2f ("mke2fs: do not warn about a pre-existing partition
table when using a non-zero offset") this behaviour was broken.
Fix this regression by explicitly creating the file if the file system
size is specified.
Samuel Holland [Sat, 29 May 2021 03:14:04 +0000 (22:14 -0500)]
libext2fs: improve error handling in POSIX ACL conversions
When encoding a POSIX ACL to the EXT4 ACL format, if an unknown tag
is encountered, that entry is silently ignored. It would be better
to return an error to inform the user that the ACL is incompatible.
Also fix the mismatched indentation in the opposite function.
Signed-off-by: Samuel Holland <samuel@sholland.org>
setup-schroot: install the acl and libreadline-dev packages
The acl package is needed to run the m_rootdir_acl test. The
libreadline-dev package will drag in the shared library package for
libreadline (libreadlineN) so that the developer running debugfs will
be able to use line editing.
libext2fs: fix translation of Posix ACL's on big-endian systems
The ACL returned by the kernel in lgetxattr(2) is returned in Little
Endian, even on Big Endian systems. Fix the functions
convert_posix_acl_to_disk_buffer() and convert_disk_buffer_to_posix_acl()
to work correctly on Big Endian systems. This fixes a failure of
the test m_rootdir_acl.
tests: try using truncate command before falling back to using dd
On the GNU Hurd, dd seems to be buggy and hangs if seeking beyond 4G.
Fortunately the GNU Hurd does have the truncate command, which does
work correctly. So try using the truncate command first, and fall
back to using dd only if the truncate command doesn't work.
e2fsck: fix f_baddotdir failure on big-endian systems
Commit 63f44aafb1f2 ("e2fsck: fix ".." more gracefully if possible")
changed the check_dot() function to try to avoid resetting the '..'
entry when the '.' entry is too large.. But if we do that, then on
big-endian systems, we need to try byte swapping the rest of the
directory entries, or else the f_baddotdir test will fail on
big-endian systems.
Also add a check to avoid UBSAN warning when there is not enough space
at the end of the directory block for a directory entry, and so we can
potentially overflow some pointer arithmetic when trying to byte swap
the remainder of the (negative) space in the directory block.
Fixes: 63f44aafb1f2 ("e2fsck: fix ".." more gracefully if possible") Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Colin Cross [Tue, 13 Jul 2021 23:29:01 +0000 (16:29 -0700)]
AOSP: Move system_shared_libs into target.bionic clause
Use target.bionic.system_shared_libs when it is used to limit the
default shared libraries (as opposed to remove them completely).
This avoids attempting to add a host dependency on libc when
system_shared_libs is modified to apply to all variants.
Also remove system_shared_libs from static binaries where it has
no effect, and consolidate it into e2fsprogs-defaults.
Eric Biggers [Wed, 16 Jun 2021 04:48:08 +0000 (21:48 -0700)]
AOSP: Use -Wno-pointer-arith in Android build
Some "arithmetic on a void pointer is a GNU extension" warnings were
introduced into upstream e2fsprogs, but they are of questionable value,
so disable them for now.
libext2fs: avoid unnecessary stat(2) calls on mountpoints
If the device name in the mtab or /proc/mounts file does not match
with the device passed into ext2fs_check_if_mounted() or
ext2fs_check_mount_point(), skip the stat(2) call on the mountpoint,
since we never use the results of the stat(2) in that case. Not only
does this provide a slight performance win, but it the stat calls on
the mountpoints could potentially trigger some SELinux denials that
could stress some sysadmins out.
David Anderson [Sat, 7 Dec 2019 01:05:54 +0000 (17:05 -0800)]
AOSP: e2fsdroid: Don't over-reserve blocks for files in Base FS.
If a large number of blocks move from one file to another file,
e2fsdroid will inadvertently reserve the moved blocks for the source
file. If the larger file is visited first, it can fail to acquire blocks
because they're reserved by the smaller file.
This patch only reserves the first N blocks of a file in Base FS, where N
is large enough to satisfy the |st_size| property from lstat(2). We only
consider "owned" blocks rather than deduplicated blocks, in case the new
file cannot be deduplicated.
mke2fs: do not warn about a pre-existing partition table when using a non-zero offset
The existing code attempted to avoid warning about a pre-existing file
system with a non-zero offset, but because the offset was not set at
the time of the check, this intention was not actually working. So
this commit will suppress warnings about pre-existing a partition
table as well as pre-existing file system when there is a non-zero
offset.
mke2fs: only try discarding a single block to test if discard works
Commit d2bfdc7ff15c ("Use punch hole as "discard" on regular files")
added a test to see if the storage device actually supports discard.
The intent was to try discarding the first block but since
io_channel_discard() interprets the offset and count arguments in
blocks, and not bytes, mke2fs was actually discarding the first 16
megabytes (when the block size is 4k). This is normally not a
problem, since most file systems are larger than that, and requests to
discard beyond the end of the block device are ignored.
However, when creating a small file system as part of a image
containing multiple partitions, the initial test discard can end up
discarding data beyond the file system being created.
Addresses-Debian-Bug: #989630 Reported-by: Josh Triplett <josh@joshtriplett.org> Fixes: d2bfdc7ff15c ("Use punch hole as "discard" on regular files") Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libe2p: use stat to prevent calling EXT2_IOC_[GS]ETFLAGS on devices
Some devices can react badly to the EXT2_IOC_[GS]ETFLAGS ioctls, since
ioctl codes are not guaranteed to be unique across different device
drivers and file systems.
Zhiqiang Liu [Wed, 30 Jun 2021 08:27:20 +0000 (16:27 +0800)]
misc: fix potential segmentation fault problem in scandir()
In scandir(), temp_list[num_dent] is allocated by calling
malloc(), we should check whether malloc() returns NULL before
accessing temp_list[num_dent].
wuguanghao [Wed, 30 Jun 2021 08:27:13 +0000 (16:27 +0800)]
profile_create_node: set magic before strdup(name) to avoid memory leak
If new->magic != PROF_MAGIC_NODE, profile_free_node() don't free node.
This will cause the node to be unable to be released correctly and
a memory leak will occur.
Signed-off-by: Wu Guanghao <wuguanghao3@huawei.com> Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com> Reviewed-by: Wu Bo <wubo40@huawei.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Wed, 16 Jun 2021 04:53:32 +0000 (21:53 -0700)]
Fix -Wunused-parameter warnings
Fix all warnings about unused function parameters that were introduced
since e2fsprogs v1.45.4, by adding EXT2FS_ATTR((unused)) or removing
parameters as appropriate.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Wed, 16 Jun 2021 04:53:31 +0000 (21:53 -0700)]
mke2fs: use ext2fs_get_device_size2() on all platforms
Since commit e8c858047be6 ("libext2fs: fix build issue for on
Windows/Cygwin systems"), ext2fs_get_device_size2() is available in
Windows builds of libext2fs. So there is no need for mke2fs to call
ext2fs_get_device_size() instead.
This fixes a -Wincompatible-pointer-types warning because
ext2fs_get_device_size() was being passed a 'blk64_t *', but it expected
a 'blk_t *'.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Wed, 16 Jun 2021 04:53:29 +0000 (21:53 -0700)]
libext2fs: improve jbd_debug() implementation
Make jbd_debug() do format string checking (but still get compiled away
to nothing) when --enable-jbd-debug isn't specified, similar to
commit d556435156b7 ("jbd2: avoid -Wempty-body warnings") on the kernel
side. This should prevent --enable-jbd-debug from getting broken due to
bad jbd_debug() statements. It also eliminates a -Wunused-variable
warning where a variable was only used in a jbd_debug() statement.
Also remove an alternative definition of jbd_debug() that was
conditional on CONFIG_JBD_DEBUG && !CONFIG_JBD_DEBUG, so was dead code.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Mon, 14 Jun 2021 21:28:30 +0000 (23:28 +0200)]
tune2fs: update overhead when toggling journal feature
When adding or removing journal from a filesystem, we also need to add /
remove journal blocks from overhead stored in the superblock. Otherwise
total number of blocks in the filesystem as reported by statfs(2) need
not match reality and could lead to odd results like negative number of
used blocks reported by df(1).
Fixes: 9046b4dfd0ce ("mke2fs: set overhead in super block") Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lukas Czerner [Mon, 14 Jun 2021 13:27:25 +0000 (15:27 +0200)]
e2fsck: fix last mount/write time when e2fsck is forced
With commit c52d930f e2fsck is no longer able to fix bad last
mount/write time by default because it is conditioned on s_checkinterval
not being zero, which it is by default.
One place where it matters is when other e2fsprogs tools require to run
full file system check before a certain operation. If the last mount
time is for any reason in future, it will not allow it to run even if
full e2fsck is ran.
Fix it by checking the last mount/write time when the e2fsck is forced,
except for the case where we know the system clock is broken.
[ Reworked the conditionals so error messages claiming that the last
write/mount time were corrupted wouldn't be always printed when the
e2fsck was run with the -f option, thus causing 299 out of 372
regression tests to fail. -- TYT ]
Fixes: c52d930f ("e2fsck: don't check for future superblock times if checkinterval == 0") Reported-by: Dusty Mabe <dustymabe@redhat.com> Signed-off-by: Lukas Czerner <lczerner@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Mon, 31 May 2021 23:31:23 +0000 (17:31 -0600)]
e2fsck: fix ".." more gracefully if possible
If the "." entry is corrupted, it will be reset in check_dot().
It is possible that the ".." entry can be recovered from the
directory block instead of also resetting it immediately. If
it appears that there is a valid ".." entry in the block, allow
that to be used, and let check_dotdot() verify the dirent itself.
When resetting the "." and ".." entries, use EXT2_FT_DIR as the
file type instead of EXT2_FT_UNKNOWN for the very common case of
filesystems with the "filetype" feature, to avoid later problems
that can be easily avoided. This can't always be done, even if
filesystems without "filetype" are totally obsolete, because many
old test images do not have this feature enabled.
Fixup affected tests using the new "repair-test" script that
updates the expect.[12] files from $test.[12].log for the given
tests and re-runs the test to ensure it now passes.
Michael Forney [Wed, 14 Apr 2021 07:41:27 +0000 (00:41 -0700)]
libext2fs: use statement-expression for container_of only on GNU-compatible compilers
Functionally, the statement-expression is not necessary here; it
just gives a bit of type-safety to make sure the pointer really
does have a compatible type with the specified member of the struct.
When statement expressions are not available, we can just use a
portable fallback macro that skips this member type check.
Signed-off-by: Michael Forney <mforney@mforney.org> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
For a large partition during e2image capture process
it is possible to overflow offset at multiply operation.
This leads to the situation when data is written to the
position at the start of the image instead of the image end.
Let's use the right cast to avoid integer overflow.
libext2fs: fix Direct I/O support in the Unix I/O manager
Commit d557b9659ba9 ("libext2fs: fix potential races in unix_io")
contained some additional changes that were supposed to clarify the
management of the length calculations when reading and writing
non-page aligned data using Direct I/O; unfortunately, those changes
were incorrect, and actually broke the Direct I/O support.
Fix these problems by reverting those portions of that commit.
[ commit description rewritten by tytso ]
Originally from https://github.com/tytso/e2fsprogs/pull/68
Fixes: d557b9659ba9 ("libext2fs: fix potential races in unix_io") Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 7 May 2021 02:40:57 +0000 (22:40 -0400)]
e2fsck: fix unaligned accesses to ext4_fc_add_range and fc_raw_inode
These fast commit related structures can be unaligned on disk. So we
need to avoid accessing these structures directly, and first copy
them to memory which we know is appropriately aligned.
This fixes an e2fsck crash while running the j_recovery_fast_commit
regression test on a sparc64 system.
e2fsck: fix unaligned accesses to ext4_fc_tl struct
Fast commit related struct ext4_fc_tl can be unaligned on disk. So,
while accessing that we should ensure that the pointers are
aligned. This patch fixes unaligned accesses to ext4_fc_tl and also
gets rid of macros fc_for_each_tl and ext4_fc_tag_val that may result
in unaligned accesses to struct ext4_fc_tl.
Theodore Ts'o [Mon, 3 May 2021 19:37:33 +0000 (15:37 -0400)]
e2fsck: fix portability problems caused by unaligned accesses
The on-disk format for the ext4 journal can have unaigned 32-bit
integers. This can happen when replaying a journal using a obsolete
checksum format (which was never popularly used, since the v3 format
replaced v2 while the metadata checksum feature was being stablized),
and in the fast commit feature (which landed in the 5.10 kernel,
although it is not enabled by default).
This commit fixes the following regression tests on some platforms
(such as running 32-bit arm architectures on a 64-bit arm kernel):
j_recover_csum2_32bit, j_recover_csum2_64bit, j_recover_fast_commit.
Andreas Dilger [Wed, 10 Mar 2021 22:47:15 +0000 (15:47 -0700)]
filefrag: minor usability improvements
Add '-V' to filefrag to print the installed version of the tool.
If '-V' is used twice, print out the list of supported FIEMAP flags.
This can be used to check if filefrag understands a specific feature.
Include FIEMAP in the error message printed when filefrag cannot
get the file layout. Since FIEMAP is commonly available and tried
first, it should also be mentioned in the error message unless it
was requested to only run FIBMAP.
Update filefrag.1.in man page to cover the new -V option.
Fix a formatting error with the recently added '-P' options, and
include '-E' and '-P' in the SYNOPSIS section.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Lustre-bug-id: https://jira.whamcloud.com/browse/LU-11848 Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Change-Id: Ib126bdd70efa1775aef6db761f54e27a593ebbe5 Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Tue, 9 Mar 2021 08:35:08 +0000 (01:35 -0700)]
e2image: add OPTIONS section to man page
Reorganize the e2image.8 man page so that the command-line options
are listed in a dedicated OPTIONS section, rather than being
interspersed among the text in the DESCRIPTION section. Otherwise,
it is difficult to determine which options are available, and to
find where each option is described.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 7 Mar 2021 04:54:33 +0000 (23:54 -0500)]
resize2fs: close the file system on errors or early exits
When resize2fs exits early, perhaps because of an error, we should
free the file system so that if MMP is in use, the MMP block is reset.
This also releases the memory to avoid memory leak reports.
Theodore Ts'o [Sun, 7 Mar 2021 04:08:12 +0000 (23:08 -0500)]
resize2fs: avoid allocating over the MMP block
When resizing past the point where the reserve inode has reserved
space for the block group descriptors to expand, and resize2fs (in an
offline resize) needs to move the allocation bitmaps and/or inode
table around, it's possible for resize2fs to allocate over the MMP
block, which would be bad.
Prevent this from happening by reserving the MMP block as a file
system metadata block (which it is) in resize2fs's accounting.
Theodore Ts'o [Sun, 28 Feb 2021 23:52:20 +0000 (18:52 -0500)]
libext2fs: fix potential races in unix_io
When unix_io does not use pread/pread64 (which is the case the bounce
buffer is in use, either when Direct I/O is in use or the
IO_FLAG_FORCE_BOUNCE in enabled), there are races between the llseek
and and read or write system calls. Fix this by using the BOUNCE_MTX
so only one thread is using the file descriptor at a time.
Theodore Ts'o [Fri, 26 Feb 2021 22:41:06 +0000 (17:41 -0500)]
libext2fs: fix unix_io's Direct I/O support
The previous Direct I/O support worked on HDD's with 512 byte logical
sector sizes, and on FreeBSD which required 4k aligned memory buffers.
However, it was incomplete and was not correctly working on HDD's with
4k logical sector sizes (aka Advanced Format Disks).
Based on a patch from Alexey Lyashkov <alexey.lyashkov@hpe.com> but
rewritten to work with the latest e2fsprogs and to minimize changes to
make it easier to review.
Theodore Ts'o [Fri, 26 Feb 2021 22:18:48 +0000 (17:18 -0500)]
iscan: fix the test program iscan so it builds again
The iscan program program isn't built by default, and was relying on
e2fsck's util.c, so it had suffered bitrot as e2fsck/util.c had
evolved. Fix it so that iscan builds correct.
Lukas Czerner [Thu, 18 Feb 2021 09:51:46 +0000 (10:51 +0100)]
mmp: do not use O_DIRECT when working with regular file
Currently the mmp block is read using O_DIRECT to avoid any caching that
may be done by the VM. However when working with regular files this
creates alignment issues when the device of the host file system has
sector size larger than the blocksize of the file system in the file
we're working with.
This can be reproduced with t_mmp_fail test when run on the device with
4k sector size because the mke2fs fails when trying to read the mmp
block.
Fix it by disabling O_DIRECT when working with regular files. I don't
think there is any risk of doing so since the file system layer, unlike
shared block device, should guarantee cache consistency.
Signed-off-by: Lukas Czerner <lczerner@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 26 Feb 2021 02:40:27 +0000 (21:40 -0500)]
resize2fs: prevent block bitmap warnings when doing extreme fs expansions
This commit fixes a bug where if a small file system is resized to
ridiculous sizes, such that the size of the resized block group
descriptor blocks exceed the original file system, and this would
result in resize2fs triggering a large number of scary warning
messages:
Illegal block number passed to ext2fs_test_block_bitmap #12440
for block bitmap for broken.img
This adds support for setting/querying the FS_NOCOMP_FL/EXT2_NOCOMPR_FL
file flag to chattr/lsattr. I picked the character "m" because it was
so far unused and all other characters that were more obvious candidates
were already taken.
The flag is available on btrfs, and with this patch it is possible to
manage it correctly.
Signed-off-by: Lennart Poettering <lennart@poettering.net> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Zhiqiang Liu [Sat, 20 Feb 2021 08:41:29 +0000 (16:41 +0800)]
debugfs: fix memory leak problem in read_list()
In read_list func, if strtoull() fails in while loop,
we will return the error code directly. Then, memory of
variable lst will be leaked without setting to *list.
e2fsck: initialize variable before first use in fast commit replay
Initialize ext2fs_ex variable in ext4_fc_replay_scan() before first
use. Also make sure ext2fs_decode_extent() completely overwrites the
extent structure passed to it as argument to prevent potential future
bugs for the users of the function.
e2fsck: add fallthrough comment in fc replay switch case
During fast commit replay scan phase, in ext4_fc_replay_scan(), we
want to fallthrough in switch case for EXT4_FC_TAG_ADD_RANGE case. Add
a comment for that.