Theodore Ts'o [Sat, 9 Nov 2019 01:26:05 +0000 (20:26 -0500)]
Sync kernel's unification of jbd2 revoke and tag block checksum handling
Commit 1101cd4d13ba ("jbd2: unify revoke and tag block checksum
handling") cleans up the fact that the jbd2_journal_revoke_tail and
jbd2_journal_block_tail structures are basically the same. So it
drops the definition of struct jbd2_journal_revoke_tail and unifies
the functions which calculates and verifies the checksums for revoke
blocks and tag blocks.
Make the same changes in e2fsprogs so eliminate unnecessary
differences in e2fsck/recovery.c and e2fsck/revoke.c.
Theodore Ts'o [Sat, 9 Nov 2019 00:00:30 +0000 (19:00 -0500)]
e2fsck/revoke.c: sync changes from kernel
Sync up the revoke.c specific changes from kernel commits 9bcf976cb8b8
("jbd2: remove unnecessary arguments of jbd2_journal_write_revoke_records"), 32ab671599a8 ("jbd2: factor out common descriptor block initialization"), 70fd76140a6c ("block,fs: use REQ_* flags directly"), cd9cb405e0b9
("jbd2: don't leak memory if setting up journal fails"), 8bdd5b60e027
("jbd2: remove NULL check before calling kmem_cache_destroy()"), 547b9ad698b4 ("jbd2: flush_descriptor(): Do not decrease buffer head's
ref count"), and fdc3ef882a5d ("jbd2: Reserve space for revoke descriptor
blocks").
Nearly all of the changes is in code under an #ifdef __KERNEL__. The
changes that will actually affect e2fprogs compilation are trivial and
easy to hand verify.
Theodore Ts'o [Fri, 8 Nov 2019 22:17:35 +0000 (17:17 -0500)]
Rename functions, types, constants to reflect jbd2 usage
We had previously stuck to using the names from ext3/jbd kernel files,
and used a script in contrib/jbd2-resync.sh to convert the kernel
files to use the ext3/jbd conventions so we could keep the files
e2fsck/recovery.c and e2fsck/revoke.c in sync with jbd2/recovery.c and
jbd2/revoke.c, respectively.
This has been getting harder and harder, so let's make a global sweep
through e2fsprogs to use the jbd2 names. Fortunately none of the
ext3/jbd names had leaked out into publically exported header files,
so this is only an internal change. Which looks scary, but it's
basically a search and replace, so if it compiles it's going to be
correct.
Theodore Ts'o [Fri, 8 Nov 2019 21:03:38 +0000 (16:03 -0500)]
Convert kernel compat functions to use new ll_rw_block() function signature
In newer kernels, ll_rw_block() separated the request operation and
the operational flags arguments. This means adding a new parameter to
ll_rw_block() (which is ignored in our compat layer) and changing READ
and WRITE to REQ_OP_READ and REQ_OP_WRITE, respectively.
This makes it easier to keep us in sync with the kernel tree.
Theodore Ts'o [Fri, 8 Nov 2019 20:02:42 +0000 (15:02 -0500)]
Clean up minor differences between kernel and e2fsck's jbd2 source files
Historically e2fsprogs's e2fsck/recovery.c and e2fsck/revoke.c was
sync'ed against the ext3 version of jbd/recovery.c and jbd/revoke.c.
Remove minor differences so we can better sync up between the two
versions.
Theodore Ts'o [Mon, 4 Nov 2019 21:43:41 +0000 (16:43 -0500)]
libext2fs: fix bug when reading or writing more than 2GB in unix_io
If count * block_size exceeds 2GB, we will overflow a 32-bit signed
integer value. This shouldn't happen in practice except for
fuzz-corrupted file systems, but let's fix the code so it's correct.
Eric Biggers [Mon, 21 Oct 2019 23:30:43 +0000 (16:30 -0700)]
AOSP: support the stable_inodes feature
Reserve the codepoint for EXT4_FEATURE_COMPAT_STABLE_INODES, allow it to
be set and cleared, and teach resize2fs to forbid shrinking the
filesystem if it is set.
This feature will allow the use of encryption policies where the inode
number is included in the IVs (initialization vectors) for encryption,
so data would be corrupted if the inodes were to be renumbered.
For more details, see the kernel patchset:
https://lkml.kernel.org/linux-fsdevel/20191021230355.23136-1-ebiggers@kernel.org/T/#u
Eric Biggers [Tue, 24 Sep 2019 21:59:10 +0000 (14:59 -0700)]
AOSP: Link to production releases, not testing releases
The testing links are not stable and are not guaranteed to exist.
Test: After re-generating the generated files, the link works now.
Change-Id: I36e73ef74571b3246f470280a75ae1098245eff5
From AOSP commit: 5971f6a5113b12b33d9454229bf27621853e1da7
Eric Biggers [Tue, 24 Sep 2019 21:59:10 +0000 (14:59 -0700)]
AOSP: Add new source files to Android.bp files
Test: see I3781b6d1e55923e9410644e8a7ba834b4d13b733
Change-Id: Ie071b30937dcf73f34df5dc3aa08d23ae30d9c63
From AOSP commit: d2a16e1616b807a6143b9c6232a1c54f90dc06f6
Theodore Ts'o [Tue, 22 Oct 2019 22:42:25 +0000 (18:42 -0400)]
Teach ext2fs_open2() to honor the EXT2_FLAG_SUPER_ONLY flag
Opening the file system with EXT2_FLAG_SUPER_ONLY will leave
fs->group_desc to be NULL and modify "dumpe2fs -h" and tune2fs when it
is emulating e2label to use this flag. This speeds up "dumpe2fs -h"
and "e2label" when operating on very large file systems.
To allow other libext2fs functions to work without too many surprises,
ext2fs_group_desc() will read in the block group descriptors on
demand. This allows "dumpe2fs -h" to be able to read the journal
inode, for example.
Eric Biggers [Wed, 18 Sep 2019 01:07:34 +0000 (18:07 -0700)]
e2fsck: check for consistent encryption policies
By design, the kernel enforces that all files in an encrypted directory
use the same encryption policy as the directory. It's not possible to
violate this constraint using syscalls. Lookups of files that violate
this constraint also fail, in case the disk was manipulated.
But this constraint can also be violated by accidental filesystem
corruption. E.g., a power cut when using ext4 without a journal might
leave new files without the encryption bit and/or xattr. Thus, it's
important that e2fsck correct this condition.
Therefore, this patch makes the following changes to e2fsck:
- During pass 1 (inode table scan), create a map from inode number to
encryption policy for all encrypted inodes. But it's optimized so
that the full xattrs aren't saved but rather only 32-bit "policy IDs",
since usually many inodes share the same encryption policy. Also, if
an encryption xattr is missing, offer to clear the encrypt flag. If
an encryption xattr is clearly corrupt, offer to clear the inode.
- During pass 2 (directory structure check), use the map to verify that
all regular files, directories, and symlinks in encrypted directories
use the directory's encryption policy. Offer to clear any directory
entries for which this isn't the case.
Add a new test "f_bad_encryption" to test the new behavior.
Due to the new checks, it was also necessary to update the existing test
"f_short_encrypted_dirent" to add an encryption xattr to the test file,
since it was missing one before, which is now considered invalid.
Google-Bug-Id: 135138675 Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Andreas Dilger <adilger@dilger.ca>
e2scrub_all: make sure fd 3 is closed before running lvm commands
Some versions of cron leave fd 3 open for some unknown reason. So
when e2scrub_all is run by cron (on non-systemd systems) this results
in an annoying message from the Cron Daemon because lvm will print
warning messages about "leaked file descriptors. So force close fd 3
at the beginning of e2scrub and e2scrub_all.
libsupport: add checks to prevent buffer overrun bugs in quota code
A maliciously corrupted file systems can trigger buffer overruns in
the quota code used by e2fsck. To fix this, add sanity checks to the
quota header fields as well as to block number references in the quota
tree.
e2fsck: make the low dtime check consistent when using the backup superblock
The backup superblock may have a last mounted time of zero, if it has
never been updated since the file system was created. In that case,
the low dtime check may get disabled when using the backup superblock,
even though subsequent e2fsck runs will end up using the low dtime
check. This can cause a failure of ext4/007, since since when e2fsck
is run a second time after the file system is mounted, the low dtime
check will trigger the e2fsck complaint:
Inode NNNN was part of the orphaned inode list. IGNORED.
Theodore Ts'o [Sun, 18 Aug 2019 23:23:07 +0000 (19:23 -0400)]
e2scrub_all: allow scrubbing in vg's whose free space == snapshot size
If the volume group's free space is exactly the same as snapshot size,
e2scrub_all will skip those logical volumes in those volume groups.
Fix this by changing the test from '>' to '>='.
Theodore Ts'o [Mon, 5 Aug 2019 16:47:18 +0000 (12:47 -0400)]
e2fsck: add a developer-only extended option: clear_all_uninit_bits
This option clears the uninitialized bit on all extents of all inodes.
Note that this can end up exposing uninitialized data to userspace.
It should only used in very specialized situations.
This option is only enabled via a new configure flag,
--enable-developer-features. It should *not* be enabled by
distributions, as it enables features thare only designed for use by
ext4 developers. These features have no documentation in the man
page, or regression tests, and if it breaks, you get to keep both
pieces.
LTO breaks reproducible builds, and there is some question as to how
reliable LTO's code generator is --- there are some scary stories that
it doesn't work well, and the GCC maintainers aren't super-interested
in fixing the bugs:
debian: skip running "make check" if DEB_BUILD_OPTIONS contains nocheck
This was done automatically by debhelper, but it got dropped when
override_dh_auto_test was added by commit 7f4c3bb120 ("debian: run
"make check" with V=1 to keep blhc happy").
debian: add a hard dependency on logsave to e2fsprogs
The initramfs created by the initramfs-tools package needs logsave and
assumes it comes along with e2fsprogs. If it is not present, the
result systems which will fail to boot. Fix this by adding the
dependency.
In the future initramfs-tools will explicitly ask for logsave (tracked
in Debian Bug: #932854), but we'll need to keep this dependency until
the next stable release of Debian.
e2scrub_all_cron: check to make sure e2scrub_all exists
Since e2scrub_all.cron is marked as a config file, it can hang around
after the package is removed, in which case e2scrub_all might not be
present. So check to make sure e2scrub_all exists before trying to
execute it.
fuse2fs: stop using the nonempty option by default
The nonempty option isn't supported by fuse3, and so if fusermount is
from fuse3, having fuse2fs specify nonempty automatically will prevent
fuse2fs from working correctly.
The strings in e2fsck/problem.c use a special %-expansion scheme,
where %b gets expanded to a block number, %i gets expanded to an inode
number, etc., where these values are in a problem context data
structure. As such, there is no need to use a printf style positional
indicator (e.g., %2$s). Indeed, the use of things like %1$i or %2$b
will cause the %-expansion code to just print %1$i or %2$b, instead of
the inode or block number, respectively.
Addresses-Debian-Bug: #892173
Signed-off-by: Theodore Ts'o <tytso@mit.edu> Cc: Philipp Thomas <pth@suse.de> Cc: Benno Schulenberg <vertaling@coevern.nl> Cc: Trần Ngọc Quân <vnwildman@gmail.com> Cc: Petr Pisar <petr.pisar@atlas.cz>
debian: drop special case CFLAGS for Alpha and PowerMac architectures
Defining HAVE_NETINET_IN_H for Alpha and __NO_STRING_INLINES for the
PowerMac QUICK bootloader date back to over two decades, to 1997 and
1998, respectively. These two architectures are no longer supported
by Debian, and it's not clear they are actually needed in 2019 even
for someone building for these architectures. So let's drop them and
see if anyone complains (or notices).
The e2scrub_all program was broken by commit c7d6525ecaab
("e2scrub_all: refactor device probe loop") so that it would use the
path of the snapshot volume instead of the base volume. This caused
"e2scrub_all -r" to pass the wrong pathname to e2scrub, with the
result that e2scrub would abort with an error instead of removing the
snapshot volume.
debian: drop support for not building the e2fsck-static and udebs packages
The ability to not build udebs packages and e2fsck-static made sense
when we were doing a separate e2fsprogs builds for those packages.
Since we're not doing that any more, we can simplify things by
dropping that flexibility.
projects / thirdparty / e2fsprogs.git / log
