Lukas Czerner [Mon, 10 Feb 2020 15:24:59 +0000 (16:24 +0100)]
tst_libext2fs: Avoid multiple definition of global variables
gcc version 10 changed the default from -fcommon to -fno-common and as a
result e2fsprogs make check tests fail because tst_libext2fs.c end up
with a build error.
This is because it defines two global variables debug_prog_name and
extra_cmds that are already defined in debugfs/debugfs.c. With -fcommon
linker was able to resolve those into the same object, however with
-fno-common it's no longer able to do it and we end up with multiple
definition errors.
Fix the problem by using SKIP_GLOBDEFS macro to skip the variables
definition in debugfs.c. Note that debug_prog_name is also defined in
lib/ext2fs/extent.c when DEBUG macro is used, but this does not work even
with older gcc versions and is never used regardless so I am not going to
bother with it.
Jeremy Visser [Mon, 3 Feb 2020 02:37:41 +0000 (13:37 +1100)]
chattr.1: improve attribute formatting with labels and indented paragraphs
By convention, lists of options in man pages use a label followed by an
indented description, such as this example from the Options section:
-R Recursively change attributes of directories and
their contents.
But the Attributes section places the available attributes mid-sentence,
which makes it visually more difficult to parse:
A file with the 'a' attribute set can only be opened
in append mode for writing. [...]
When a file with the 'A' attribute set is accessed, its
atime record is not modified. [...]
This patch places a label beside each attribute description, which (in
my opinion) improves readability, especially when visually skimming the
list. For example:
a A file with the 'a' attribute set can only be
opened in append mode for writing.
A When a file with the 'A' attribute set is accessed,
its atime record is not modified.
Signed-off-by: Jeremy Visser <jeremyvisser@google.com> Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If the bad block list has been reset in the middle of an inode scan,
it's possible for bb->list[scan->bad_blocks_ptr] to result in an
out-of-bounds read access.
This is highly unlikely to happen under normal circumstances; in
particular, we generally don't use bad block inodes any more. In
addition, this would only happen if the bad block inode itself is
corrupt so e2fsck needs to wipe it out. This might cause e2fsck to
crash, but it will more likely cause a part of the inode table to be
wrongly considered invalid, causing file system to be incorrectly
fixed.
This was reported by TALOS as TALOS-2020-0974 and CVE-2020-6057, but
after closer examination, we don't believe this can be used in any way
to exploit the system or release information about the system, since
all this can do is to cause part of the inode table to be skipped when
it shouldn't be, and this can't be leveraged since any information
about the ASLR of the process is obsolete once e2fsck exits.
Andreas Dilger [Tue, 14 Jan 2020 21:42:18 +0000 (14:42 -0700)]
mmp: abstract out repeated 'sizeof(buf), buf' usage
The printf("%.*s") format requires both the buffer size and buffer
pointer to be specified for each use. Since this is repeatedly given
as "(int)sizeof(buf), (char *)buf" for mmp_nodename and mmp_bdevname
fields, with typecasts to avoid compiler warnings.
Add a helper macro EXT2_LEN_STR() to avoid repeated boilerplate code.
This can also be used for other superblock buffer fields that may not
have NUL-terminated strings (e.g. s_volume_name, s_last_mounted,
s_{first,last}_error_func, s_mount_opts) to simplify code and avoid
the need for temporary buffers for NUL-termination.
Annotate the superblock string fields that may not be NUL-terminated.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Tue, 14 Jan 2020 21:42:17 +0000 (14:42 -0700)]
mmp: don't assume NUL termination for MMP strings
Don't assume that mmp_nodename and mmp_bdevname are NUL terminated,
since very long node/device names may completely fill the buffers.
Limit string printing to the maximum buffer size for safety, and
change the field definitions to __u8 to make it more clear that
they are not NUL-terminated strings, as is done with other strings
in the superblock that do not have NUL termination.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 16 Jan 2020 20:35:29 +0000 (15:35 -0500)]
libext2fs: reserve the error code EXT2_ET_NO_GDESC
This is really only needed in the 1.46+ where the EXT2_FLAG_SUPER_ONLY
is honored by ext2fs_open to only read the superblock, so that
fs->group_desc can be NULL. We define it in the maint branch so that
we can be sure the error tables are kept in sync (in the unlikely case
that a new error code needs to be assigned in the maint branch).
Theodore Ts'o [Mon, 6 Jan 2020 23:01:48 +0000 (18:01 -0500)]
libext2fs: always compile swapfs functions on all architectures
By only compiling the ext2fs_swap_* functions on big-endian systems,
it causes debian/libext2fs2.symbols to need to be different on
different little-endian vs big-endian architectures. Including the
ext2fS_swap_* functions increases the size of the library by ~6k,
which is not a big deal.
Theodore Ts'o [Mon, 6 Jan 2020 21:01:23 +0000 (16:01 -0500)]
e2scrub, e2scrub_all: don't sleep unnecessarily in exitcode
The two second sleep is only needed in e2scrub, and when there is a
failure, so that systemd has a chance to gather the log output before
e2scrub exits. It's not needed if the script is exiting successfully,
and it's never needed for e2scrub_all ever.
Theodore Ts'o [Mon, 6 Jan 2020 19:42:01 +0000 (14:42 -0500)]
libext2fs: don't scan /etc/mtab if file system not found in /proc/mounts
Previously we would scan /etc/mtab if the device is not found in
/proc/mounts. This is because previously, /etc/mtab would have the
filename for a loopback mount, while /proc/mounts would only have
something like /dev/loop0. Since on many systems /etc/mtab is now a
symlink to /proc/mounts, ismounted.c has a special function,
check_loop_mounted.
For this reason, it's not necessary to fall back to trying to scan
/etc/mtab if a device / filename is not found from scanning
/proc/mounts. This also prevents failures if the file /etc/mtab does
not exist but /proc/mounts does exist when checking to see if a device
is mounted when it isn't.
Theodore Ts'o [Thu, 2 Jan 2020 02:50:27 +0000 (21:50 -0500)]
e2fsck: don't check for future superblock times if checkinterval == 0
We are no longer enabling periodic file system checks by default in
mke2fs. The only reason why we force file system checks if the last
mount time or last write time in the superblock is if this might
bypass the periodic file systme checks. So if the checkinterval is
zero, skip the last mount/write time checks since there's no reason to
force a check just because the system clock is incorrect.
Theodore Ts'o [Wed, 1 Jan 2020 00:48:37 +0000 (19:48 -0500)]
Drop use of -pedantic when doing gcc-wall
With newer versions of gcc -pedantic is *super* pedantic, and
generates way too much noise. So we drop it, and thus we don't need
util/gcc-wall-cleanup and util/static-analysis-cleanup.
Theodore Ts'o [Wed, 1 Jan 2020 00:00:49 +0000 (19:00 -0500)]
libext2fs: chage ext2_off_t and ext2_off64_t to be signed types
The ext2_off_t and ext2_off64_t types are used by ext2_file_lseek()
and ext2_file_llseek(), and they need to be signed so that it can be a
negative offset from the end of the file when EXT2_SEEK_END is used.
Theodore Ts'o [Tue, 31 Dec 2019 22:16:25 +0000 (17:16 -0500)]
libext2fs: use __u64 instead of ext2_off64_t in qcow.c
The type ext2_off64_t should only be used in the context of the ext2fs
library's llseek() and ftruncate() analogs, ext2_file_llseek() and
ext2_inode_set_size(). That's because like the POSIX.1's off_t, it
needs to be a signed integer value so that SEEK_END will work
correctly.
qcow.c's use of ext2_off64_t is only internal, and not in any exposed
interfaces, so changes nothing; it just prepares us to be able to
change the type of ext2_off64_t.
Li Dongyang [Wed, 20 Nov 2019 04:35:24 +0000 (04:35 +0000)]
mke2fs: fix setting bad blocks in the block bitmap
We mark the bad blocks as used on fs->block_map before allocating
group tables. Don't translate the block number to cluster number when
doing this, the fs->block_map is still a block-granularity allocation
map, it will be coverted later by ext2fs_convert_subcluster_bitmap().
Signed-off-by: Li Dongyang <dongyangli@ddn.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
David Anderson [Sat, 7 Dec 2019 00:48:52 +0000 (16:48 -0800)]
AOSP: e2fsdroid: Skip Base FS entries that no longer exist.
Don't reserve blocks in the base map if the file does not exist in the
target image. This can happen if a file is removed or renamed in between
two builds. If the removed file is quite large, skipping it is important
since otherwise it will prevent blocks from being allocated for new files.
David Anderson [Thu, 5 Dec 2019 22:00:28 +0000 (14:00 -0800)]
AOSP: e2fsdroid: Allow re-use of deduplicated blocks.
When using a Base FS map, track deduplicated blocks in a separate
bitmap. The first inode to request a block from this set will succeed
in getting the block. Blocks in the dedup set are not available for
libext2fs to allocate; this ensures that previously deduplicated blocks
are re-used for their original purpose.
Note that deduplication takes priority over block allocation, so that
once a block is removed from the dedup set, that does not actually
prevent it from being re-used. Similarly, a file that was not previously
sharing a block may have its blocks shared in the new image.
David Anderson [Thu, 5 Dec 2019 21:27:44 +0000 (13:27 -0800)]
AOSP: e2fsdroid: Do not allocate invalid blocks from BaseFS.
If certain metadata properties change in between builds, such as the
inode table size, then block mappings may not be valid from one build to
the next. For example, build A could allocate block N for a file. If
build B has a larger inode table, block N may no longer be a data block.
In this case, we need to remove the block from the BaseFS range list so
we do not give it back to libext2fs for writing data. We do this with a
new "owned_block_map" bitmap. If a block from the base FS is not in use
by the initial image, and is not used by any other file, it is
considered owned, and can be claimed by that file.
Note that while this produces correct images, it also prevents
deduplicated blocks from being re-used. This will be addressed in a
follow-up patch.
Bug: 145316683
Test: e2fsdroid block mapping with dynamic partitions
Change-Id: I3145e45156f7879bdf956384723fab4bd69acb93 Signed-off-by: David Anderson <dvander@google.com>
From AOSP commit: b3d1ccdb673772588b7bb14c4581980ef549a0b8
David Anderson [Thu, 5 Dec 2019 20:23:12 +0000 (12:23 -0800)]
AOSP: e2fsdroid: Refactor basefs allocation.
This refactors base_fs_alloc_load() to address two issues. First, the
failure cases have been made simpler by factoring out a common helper
for freeing a base_fs_allocator. Second, we no longer return
EXIT_FAILURE, since this is not an errcode_t.
David Anderson [Thu, 5 Dec 2019 01:38:11 +0000 (17:38 -0800)]
AOSP: e2fsdroid: Refactor block_range.
block_range is a singly-linked list, but the head/tail links are
manually managed all over. Instead, introduce a block_range_list
structure and refactor list helpers to operate on this instead. This
ensures head/tail are maintained properly (in some cases, like
delete_block_range, they were not).
Wang Shilong [Tue, 31 Dec 2019 00:52:39 +0000 (19:52 -0500)]
e2fsck: fix use after free in calculate_tree()
The problem is alloc_blocks() will call get_next_block() which might
reallocate outdir->buf, and memory address could be changed after
this. To fix this, pointers that point into outdir->buf, such as
int_limit and root need to be recaulated based on the new starting
address of outdir->buf.
[ Changed to correctly recalculate int_limit, and to optimize how we
reallocate outdir->buf. -TYT ]
Signed-off-by: Wang Shilong <wshilong@ddn.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 30 Dec 2019 15:12:58 +0000 (10:12 -0500)]
ext2fs: add ext2fs_get_stat_i_blocks() function
The function ext2fs_inode_i_blocks() is a bit confusing whether it is
returning the inode's i_blocks value, or whether it is returning the
value ala the stat(2) system call, which returns i_blocks in units of
512 byte sectors. This caused ext2fs_inode_i_blocks() to be
incorrectly used in fuse2fs and the function quota_compute_usage().
To address this, we add a new function, ext2fs_get_stat_i_blocks()
which is clearly labelled what it is returning, and use it in fuse2fs
and quota_compute_usage(). It's also a bit more convenient to use it
in e2fsck, so use it there too.
Reported-by: Wang Shilong <wangshilong1991@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 20 Dec 2019 00:45:06 +0000 (19:45 -0500)]
e2fsck: don't try to rehash a deleted directory
If directory has been deleted in pass1[bcd] processing, then we
shouldn't try to rehash the directory in pass 3a when we try to
rehash/reoptimize directories.
Theodore Ts'o [Fri, 20 Dec 2019 00:37:34 +0000 (19:37 -0500)]
e2fsck: abort if there is a corrupted directory block when rehashing
In e2fsck pass 3a, when we are rehashing directories, at least in
theory, all of the directories should have had corruptions with
respect to directory entry structure fixed. However, it's possible
(for example, if the user declined a fix) that we can reach this stage
of processing with a corrupted directory entries.
So check for that case and don't try to process a corrupted directory
block so we don't run into trouble in mutate_name() if there is a
zero-length file name.
Theodore Ts'o [Sat, 14 Dec 2019 04:30:53 +0000 (23:30 -0500)]
e2fsck: optimize away repeated calls to gettext()
Optimize _("getting next inode from scan") so it is not called for
each initialized inode in the file system, and make a similar
optimization in pass 2 for each directory block.
Theodore Ts'o [Sat, 23 Nov 2019 04:27:47 +0000 (23:27 -0500)]
Add a program to test images provided by UBSAN fuzzing reports
This program calls a few ext2fs library functions used by the current
generation of libext2fs fuzzers, and is helpful in reproducing UBSAN
failures reported externally.
Darrick J. Wong [Wed, 20 Nov 2019 19:32:55 +0000 (11:32 -0800)]
tune2fs: prohibit toggling uninit_bg on live filesystems
An internal customer followed an erroneous AskUbuntu article[1] to try to
change the UUID of a live ext4 filesystem. The article claims that you
can work around tune2fs' "cannot change UUID on live fs" error by
disabling uninit_bg, changing the UUID, and re-enabling the feature.
This led to metadata corruption because tune2fs' journal descriptor
rewrite races with regular filesystem writes. Therefore, prevent
administrators from turning on or off uninit_bg on a mounted fs.
Eric Biggers [Mon, 18 Nov 2019 01:48:48 +0000 (17:48 -0800)]
chattr.1: adjust documentation for encryption attribute
Adjust the documentation for the encryption attribute ('E') to clarify
that encryption isn't experimental anymore and isn't restricted to
regular files, and that the encryption is done by the filesystem.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 13 Nov 2019 18:55:29 +0000 (13:55 -0500)]
configure: don't fail if /usr/include/sys/mount.h does not exist
The AX_CHECK_MOUNT_OPT macro fails if /usr/include/sys/mount.h.
Unfortunately, the GNU HURD doesn't have this header file. Drop the
requirement, since if it doesn't exist, the macro will assume that
given mount options don't exist, which for e2fsprogs is used to test
to for the existence of the mount options nosuid and nodev. This is
only used for fuse2fs, and HURD doesn't support fuse2fs anyway.
Theodore Ts'o [Sun, 10 Nov 2019 17:11:49 +0000 (12:11 -0500)]
libext2fs/ismounted.c: check device id in advance to skip false device names
If there is a trickster which tries to use device names as the mount
device for pseudo-file systems, the resulting /proc/mounts can confuse
ext2fs_check_mount_point(). (So far as I can tell, there's no good
reason to do this, but sysadmins do the darnest things.)
An example of this might be the following /proc/mounts excerpt:
This is created via "mount -t tmpfs /dev/sdb /mnt2" followed via
"mount -t ext4 /dev/sdb /mnt". (Normally, a sane mount of tmpfs would
use something like "mount -t tmpfs tmpfs /mnt2".)
Fix this by double checking the st_rdev of the claimed mountpoint and
match it with the dev_t of the device. (Note that the GNU HURD
doesn't support st_rdev, so we can't solve this problem for the HURD.)
Jan Kara [Fri, 18 Oct 2019 12:50:59 +0000 (14:50 +0200)]
resize2fs: make minimum size estimates more reliable for mounted fs
Currently, the estimate of minimum filesystem size is using free blocks
counter in the superblock. The counter generally doesn't get updated
while the filesystem is mounted and thus the estimate is very unreliable
for a mounted filesystem. For some usecases such as automated
partitioning proposal to the user it is desirable that the estimate of
minimum filesystem size is reasonably accurate even for a mounted
filesystem. So use group descriptor counters of free blocks for the
estimate of minimum filesystem size. These get updated together with
block being allocated and so the resulting estimate is more accurate.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 5 Nov 2019 01:54:20 +0000 (17:54 -0800)]
e2scrub_all: fix broken stdin redirection
gregor herrmann reports that the weekly e2scrub cronjob emits these
errors:
/sbin/e2scrub_all: line 173: /proc/8234/fd/pipe:[90083173]: No such file or directory
The root cause of this is that the ls_targets stdout is piped to stdin
to the entire ls_targets loop body to prevent the loop body from reading
the loop iteration items. Remove all the broken hackery by reading the
target list into a bash array and iterating the bash array.
Addresses-Debian-Bug: #944033
Reported-by: gregor herrmann <gregoa@debian.org> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 5 Nov 2019 01:54:14 +0000 (17:54 -0800)]
e2scrub_all: don't even reap if the config file doesn't allow it
Dave Chinner complains that the automated on-boot e2scrub reaping takes
a long time (because the lvs command can take a while to run) even
though the automated e2scrub is disabled via e2scrub.conf on his
systems.
We still need the reaping service to kill off stale e2scrub snapshots
after a crash, but it's unnecessary to annoy everyone with slow bootup.
Because we can look for the e2scrub snapshots in /dev/mapper, let's
skip reaping if periodic e2scrub is disabled unless we find evidence of
e2scrub snapshots in /dev.
Reported-by: Dave Chinner <david@fromorbit.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 4 Nov 2019 21:43:41 +0000 (16:43 -0500)]
libext2fs: fix bug when reading or writing more than 2GB in unix_io
If count * block_size exceeds 2GB, we will overflow a 32-bit signed
integer value. This shouldn't happen in practice except for
fuzz-corrupted file systems, but let's fix the code so it's correct.
Eric Biggers [Tue, 24 Sep 2019 21:59:10 +0000 (14:59 -0700)]
AOSP: Link to production releases, not testing releases
The testing links are not stable and are not guaranteed to exist.
Test: After re-generating the generated files, the link works now.
Change-Id: I36e73ef74571b3246f470280a75ae1098245eff5
From AOSP commit: 5971f6a5113b12b33d9454229bf27621853e1da7
Eric Biggers [Tue, 24 Sep 2019 21:59:10 +0000 (14:59 -0700)]
AOSP: Add new source files to Android.bp files
Test: see I3781b6d1e55923e9410644e8a7ba834b4d13b733
Change-Id: Ie071b30937dcf73f34df5dc3aa08d23ae30d9c63
From AOSP commit: d2a16e1616b807a6143b9c6232a1c54f90dc06f6
e2scrub_all: make sure fd 3 is closed before running lvm commands
Some versions of cron leave fd 3 open for some unknown reason. So
when e2scrub_all is run by cron (on non-systemd systems) this results
in an annoying message from the Cron Daemon because lvm will print
warning messages about "leaked file descriptors. So force close fd 3
at the beginning of e2scrub and e2scrub_all.
libsupport: add checks to prevent buffer overrun bugs in quota code
A maliciously corrupted file systems can trigger buffer overruns in
the quota code used by e2fsck. To fix this, add sanity checks to the
quota header fields as well as to block number references in the quota
tree.
e2fsck: make the low dtime check consistent when using the backup superblock
The backup superblock may have a last mounted time of zero, if it has
never been updated since the file system was created. In that case,
the low dtime check may get disabled when using the backup superblock,
even though subsequent e2fsck runs will end up using the low dtime
check. This can cause a failure of ext4/007, since since when e2fsck
is run a second time after the file system is mounted, the low dtime
check will trigger the e2fsck complaint:
Inode NNNN was part of the orphaned inode list. IGNORED.
Theodore Ts'o [Sun, 18 Aug 2019 23:23:07 +0000 (19:23 -0400)]
e2scrub_all: allow scrubbing in vg's whose free space == snapshot size
If the volume group's free space is exactly the same as snapshot size,
e2scrub_all will skip those logical volumes in those volume groups.
Fix this by changing the test from '>' to '>='.
LTO breaks reproducible builds, and there is some question as to how
reliable LTO's code generator is --- there are some scary stories that
it doesn't work well, and the GCC maintainers aren't super-interested
in fixing the bugs:
debian: skip running "make check" if DEB_BUILD_OPTIONS contains nocheck
This was done automatically by debhelper, but it got dropped when
override_dh_auto_test was added by commit 7f4c3bb120 ("debian: run
"make check" with V=1 to keep blhc happy").
debian: add a hard dependency on logsave to e2fsprogs
The initramfs created by the initramfs-tools package needs logsave and
assumes it comes along with e2fsprogs. If it is not present, the
result systems which will fail to boot. Fix this by adding the
dependency.
In the future initramfs-tools will explicitly ask for logsave (tracked
in Debian Bug: #932854), but we'll need to keep this dependency until
the next stable release of Debian.
e2scrub_all_cron: check to make sure e2scrub_all exists
Since e2scrub_all.cron is marked as a config file, it can hang around
after the package is removed, in which case e2scrub_all might not be
present. So check to make sure e2scrub_all exists before trying to
execute it.
projects / thirdparty / e2fsprogs.git / log
