Theodore Ts'o [Tue, 21 May 2024 02:52:47 +0000 (22:52 -0400)]
ext4.5: add preprocessor hint
This fixes a Lintian warning which is triggered by an arbtrary
MANROFFSEQ='' environment variable:
an.tmac:<standard input>:376: warning: tbl preprocessor failed, or it or soelim was not run; table(s) likely not rendered (TE macro called with TW register undefined)
Jiyong Park [Wed, 1 May 2024 07:17:36 +0000 (16:17 +0900)]
AOSP: Use no_full_install: true instead of installable: false
So far, we have used `instalable: false` to avoid collision with the
other modules that are installed to the same path. A typical example was
<foo> and <foo>.microdroid. The latter is a modified version of the
former for the inclusion of the microdroid image. They however both have
the same instalation path (ex: system/bin) and stem (ex: foo) so that we
can reference them using the same path regardless of whether we are in
Android or microdroid.
However, the use of `installable: false` for the purpose is actually
incorrect, because `installable: false` also means, obviously, "this
module shouldn't be installed". The only reason this incorrect way has
worked is simply because packaging modules (ex: android_filesystem)
didn't respect the property when gathering the modules.
As packaging modules are now fixed to respect `installable: false`, we
need a correct way of avoiding the collision. `no_full_install: true` is
it.
If a module has this property set to true, it is never installed to the
full instal path like out/target/product/<partition>/... It can be
installed only via packaging modules.
Eric Biggers [Thu, 23 Mar 2023 02:52:09 +0000 (02:52 +0000)]
AOSP: ext2simg: fix same_file() with symlinks
Fix same_file() to use stat() instead of lstat() when checking the
paths, so that symlinks are dereferenced. This is needed to be
consistent with how the paths are actually accessed later. Otherwise,
not all cases where the input and output file are the same are detected.
Also just use the stat() result to check whether the output file exists,
instead of using a separate call to access().
Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool")
Change-Id: Ie36981f9dbc19494732f518488a75fb92c0f0343 Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 08c122f12fc231029a74c24b969e337203c7b6e2
Eric Biggers [Thu, 23 Mar 2023 00:44:22 +0000 (00:44 +0000)]
AOSP: ext2simg: clean up integer types and check for too-large fs
libsparse assumes 32-bit block numbers. Also, ext2simg might read
nearly the entire filesystem into memory.
Therefore, make ext2simg use appropriate integer types, and explicitly
check for when the filesystem is too large or allocating memory failed.
Change-Id: Ic415d0e974dce2b4ff6e7fa9265f6e86d371a274 Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 8fff11068c100be627745967992fb88759dea9c1
Eric Biggers [Thu, 23 Mar 2023 00:44:22 +0000 (00:44 +0000)]
AOSP: ext2simg: clean up add_chunk()
Remove a level of indentation, check a bool in the normal way, and
simplify the linked list handling. No change in behavior.
Change-Id: I12589a254f155b1c40418458a666b87c7ef5c1cf Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 7d0f5c1aca332da22e4878f5825e0ffb5122f96b
Eric Biggers [Thu, 23 Mar 2023 00:44:21 +0000 (00:44 +0000)]
AOSP: ext2simg: use a standard flexible array
Use a standard flexible array instead of a nonstandard zero-length
array. No change in behavior.
Change-Id: Ifdce24f5d6e2471634bb785527def3fe8fefc202 Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: c88ea796fbf7f4c79155196ec483681b3733bbff
Eric Biggers [Thu, 23 Mar 2023 00:44:21 +0000 (00:44 +0000)]
AOSP: ext2simg: use bool where appropriate
For the values that get used as the 'bool' parameters of
sparse_file_write(), use 'bool' in ext2simg too. No change in behavior.
Change-Id: I05f7d6fd3027eb10231c035f9fdc8e946e2c4c90 Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 2728c6e766976acbf442d3721f2d93960e13682e
The chunk_end parameter to add_chunk() is exclusive, but two callers
incorrectly treat it as inclusive: when the maximum chunk length of
'INT32_MAX - 12' bytes is reached, and when a chunk extends to the very
end of the filesystem. The result is that the output simg file contains
zeroes for the last block of these chunks instead of the correct data.
A related bug is that the expanded size of the simg file is set to the
filesystem size (in blocks) minus s_first_data_block. On filesystems
where s_first_data_block != 0, i.e. 1K blocksize filesystems without
bigalloc enabled, this truncates the last block of the filesystem.
Fix these bugs by (a) making add_chunk() take the chunk length and
passing the correct values, and (b) using the filesystem size properly.
Here is a reproducer that shows the last block of the filesystem being
truncated (bsize=1024) and being corrupted with zeroes (bsize=4096):
Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool") Reported-by: Clemens Lang <clemens.lang@bmw.de>
Change-Id: I3b64c4fbffa5821b431f29e99b36168617da7563 Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 1e498908c6ac13b4d5ec0117f4ddcd577aac607e
Eric Biggers [Wed, 8 Feb 2023 21:42:02 +0000 (21:42 +0000)]
AOSP: Mostly restore -Werror for macOS build
It turns out the "Can't use getmntent or getmntinfo" warning was the
only warning remaining in the macOS build via the Android build system.
So now that it's fixed, -Wno-error can be removed.
That being said, the upstream CI (GitHub Actions) currently uses
-Wno-error=deprecated-declarations for the macOS build, since it's still
needed for some files (which aren't built by the Android build system).
For now, let's just replace -Wno-error with
-Wno-error=deprecated-declarations to match what the upstream CI uses.
Eric Biggers [Thu, 2 Feb 2023 17:56:20 +0000 (17:56 +0000)]
AOSP: Revert "Android: stop suppressing warnings from macOS build"
Unfortunately, the macOS build is not tested either by presubmit or by
local builds. A macOS build *is* being tested in the upstream GitHub
Actions workflow now; however, that uses the autotools-based build
system, and there can be issues specific to the Android build system.
As a result, removing -Wno-error was not safe yet, and the macOS build
is currently broken in postsubmit. As there could be multiple issues,
let's restore -Wno-error until I've had a chance to fix the warnings.
Theodore Ts'o [Sat, 18 May 2024 04:56:58 +0000 (00:56 -0400)]
libext2fs: add ext2_types.h to qcow2.h
The qcow2.h header file uses types such __u32 which are defined in
ext2_types.h. So include it directly to avoid relying on users of the
qcow2.h header file to include right dependencies.
Jan Kara [Mon, 6 May 2024 17:41:19 +0000 (19:41 +0200)]
e2fsck: fix golden output of several tests
Some old tests of EA inodes were not in fact completely fixing the
filesystem (like they were leaving directories with EA_INODE_FL set or
EA inodes referenced from directory hierarchy). New e2fsck checks fix
these so golden output changes. Update it.
Jan Kara [Mon, 6 May 2024 17:41:17 +0000 (19:41 +0200)]
e2fsck: add more checks for ea inode consistency
Currently checking of EA inodes was rather weak. Add several more
consistency checks.
1) Check that EA inode is a regular file.
2) Check that EA_INODE feature is set if the filesystem has EA inodes.
3) Make sure that no EA inode is referenced from directory hierarchy.
Above issue can reproduce as follows:
step1: socat UNIX-LISTEN:/home/test/mysocket.sock,mode=777,reuseaddr,fork EXEC:/home/test &
step2: setfacl some xattr for mysocket.sock
step3: cp -a /home/test/mysocket.sock /home/test/sock1
cp -a /home/test/mysocket.sock /home/test/sock2
step4: sync
step5: Power-off
step6: run e2fsck
As after commit 42475e281d22 add ext2fs_inode_has_valid_blocks() judgement in
release_inode_blocks() which means socket type file skip realse block include
ACL block. The kernel does not restrict the setting of extended attributes for
socket files. So this will lead to ACL block leak.
To solve above issue there's need to release ACL block for other kind of
special file.
Fixes: 42475e281d22 ("super.c (release_inode_blocks): Don't try to release the blocks if the orphaned inode is a device file, symlink, or some other kind of special file that doesn't have a block list.") Signed-off-by: Ye Bin <yebin10@huawei.com> Reviewed-by: Jan Kara <jack@suse.cz> Link: https://lore.kernel.org/r/20240418063946.2802835-1-yebin10@huawei.com Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 2 May 2024 16:23:44 +0000 (12:23 -0400)]
libext2fs: fix potential divide by zero bug caused by a lxcfs bug
If sysconf(_SC_NPROCESSORS_CONF) returns zero, this can cause a divide
by zero. Make ext2fs_rw_bitmaps() more robust defaulting to 4 threads
if _SC_NPROCESSORS_CONF returns an invalid value.
Theodore Ts'o [Wed, 1 May 2024 21:22:55 +0000 (17:22 -0400)]
e4defrag: use snprintf to assure that there can't be a buffer overflow
The size of msg_buffer is carefully calculated so it can never
overflow, but it triggers a Coverity warning. Use snprintf instead of
sprintf to silence the Coverity warning.
Theodore Ts'o [Wed, 1 May 2024 20:58:50 +0000 (16:58 -0400)]
libsupport: use explicit type widths instead of time_t
The in-memory data structures used time_t for the grace period (which
is a delta timestamp denominated in seconds), as well as the soft
limit expiration time (which is an actual time_t). Use an explicit
__u32 for the former, and the __u64 for the latter.
This silences a Coverity warning, but more importantly, using an
explicit __u64 for the expiration time means that running e2fsck on a
platform with a 32-bit time_t, and it needs to read and then modify a
quota structure, we won't lose the high 32-bits of the quota
expiration time.
Theodore Ts'o [Wed, 1 May 2024 04:24:52 +0000 (00:24 -0400)]
e2fsck: check the error return from the forced rewrite write
If read of a block fails, we offer the user the opportunity to force a
rewrite to that sector to force the storage device to remap the LBA to
its spare block pool. Check that write so if it fails, we can let the
user know.
Theodore Ts'o [Wed, 1 May 2024 04:20:10 +0000 (00:20 -0400)]
resize2fs: mark that the error return is deliberately ignored
When moving the inode table, if writing the (partially overlapping)
inode table fails, we need to write it back in its original location
before bailing out. If that write unding the initial write fails,
there's nothing we can do, so we ignore it. Mark this to avoid a
false positive from Coverity.
Theodore Ts'o [Wed, 1 May 2024 03:54:26 +0000 (23:54 -0400)]
e2scrub: test for the presence of systemd using test -e /run/systemd/system
Debian has a package called "systemctl" which provides a systemctl
executable to "manage services without systemd". So test for whether
we have a fully functional systemd system by checking for the
existence of /run/systemd/system instead testing for the presence of
the command named systemctl.
The problem with explicitly setting _FILE_OFFSET_BITS is that
it's not necessarily a no-op on a 64-bit platform with a 64-bit off_t.
Apparently glibc's mips64el which end up using a different structure
definition for struct stat, and this causes a compatibility problem
with libarchive. It's not needed on mips64el, since off_t is 64-bits,
but it actually causes problems.
So remove it, since we now use the autoconf's AC_SYS_LARGEFILE, which
will set _FILE_OFFSET_BITS when it is necessary (such as on a 32-bit
i386 Linux platform), and will skip it when it is unnecessary.
The libarchive functionality in "mke2fs -d foo.tar" is breaking the
regression test[1]. Since this is working everywhere _except_
mips64el, as a short-term workaround disable libarchive support on
this platform until it can be fixed.
The e2scrub scripts rely on systemd, which isn't present on non-Linux
systems, so they aren't built. So we need to skip trying to run
dh_installsystemd since it will fail on the Hurd build since the
requisite files aren't being built.
Teach configure the --without-libarchive option, which forcibly
disables use of the libarchive library.
The option --with-libarchive=direct will disable the use of dlopen,
and will link mke2fs with -larchive directly. This doesn't work when
building mke2f.static, since -larchive has a large number of
depedencies, and even "pkgconf --libs --static libarchive" doesn't
provide all of the appropriate library dependencies. :-(
debian: add a note in debian/changelog regarding features being re-enabled
The metadata_csum_seed and orphan_file features were disabled before
Debian Bookworm was released, but now that it's released, we are now
re-enabling those features for Debian testing and the next version of
Debian stable (trixie).
Manually count the number free clusters in the last block group since
it might not be a multiple of 8, and using ext2fs_bitcount() might not
work if bitmap isn't properly padding out.
In addition, when setting up the block bitmap for the resized file
system, resize2fs was setting up the "real end" of the bitmap in units
of blocks instead of clusters.
We didn't notice this problem earlier because of a test failure which
caused the test to be skipped.
Prevent i_dtime from being mistaken for an inode number post-2038 wraparound
We explicitly decided not to reserve space for a 64-bit dtime, since
it's never displayed or exposed to userspace. The dtime field is used
a linked list for the ophan list, and for forensic purposes when
trying to determine when an inode was deleted. So right after the
2038 epoch, a deleted inode might end up with a dtime which is zero or
smaller than the number of inodes, which will result in e2fsck
reporting a potential problems. So when we set the dtime, make sure
that the dtime won't be mistaken for an inode number.
libextr2fs: handle short reads/writes while creating the qcow file
This issue was flagged by Coverity, although its analysis was
incorrect. This isn't actually a memory overrun / security issue, but
rather a functional correctness issue since POSIX allows reads and
writes to be partially completed, and in those cases qcow2_copy_data()
could result in a corrutped qcow2 file.
configure: Use FORTIFY_SOURCE=3 when hardening is enabled
FORTIFY_SOURCE=3 provides much more robust checks for buffer overruns
and other memory bugs[1]. It requires gcc 12 and glibc 2.34 which
should be available on most modern distributions (which are the ones
that use --enable-hardening).
mke2fs: implement timestamp clamping if SOURCE_DATE_EPOCH is set
When copying files to the newly created file system using "mke2fs -d",
and there are timestamps greater than what is specified by
SOURCE_DATE_EPOCH, clamp the timestamp to the SOURCE_DATE_EPOCH
timestamp.
libext2fs: use a safe_getenv() function everywhere
Hoist safe_getenv() from test_io.c and unix_io.c to a globally
exported ext2fs_safe_getenv() and use it instead of getenv() in
libext2fs. This provides a bit more safety if e2fsprogs programs are
used in setuid contexts.
Fix coverity false positives introduced by the post-2038 changes
Commit ca8bc9240a00 ("Add post-2038 timestamp support...") did things
like casting a 64-bit unsigned integer into a signed 32-bit integer
deliberately; but Coverity thinks this is a bug. So mask off the bits
to make it clear this was deliberate.
e2fsck: make sure get_backup_sb() works when ctx is NULL
The print_e2fsck_message() function can call get_backup_sb() with the
ctx variable set to NULL. In that case, we can't dereference
ctx->filesystem_name; instead, we can get the size of the file system
from the ext2fs_block_count(fs->super).
Align function prototypes for libss's request handler function
Clang 17's Undefined Behaviour Sanitizer will throw run-time warnings
if a function pointer is dereferenced with a different function
signature than one in the pointer --- even if the difference is a
missing const qualifier. To fix regression test failures, change
declarations of argv to use ss_argv_t instead of an inconsistently
open-coded type.
The mkgnutar.pl file only works if the developer had a specific
username and uid. In addition, if it is used, the round-trip from tar
to an ext4 file system and back to tar isn't properly tested. So only
use mkgnutar.pl if the system doesn't have GNU TAR.
In addition, make sure all of the temp files created by the test are
deleted when the test is completed.
FreeBSD 14 has changed the definition of qsort_r to align it with the
POSIX, but it did this with a #define. So when sort_r.h tries to
provide a function prototype, surround the function name with
parenthesis so it doesn't get expanded by FreeBSD's #define.
Debugfs's stat command called ext2fs_inode_xtime_get() with a struct
inode * instead of a struct large_inode *. As a result, printing
inode timestamps will be incorrect if the time value is larger than
2**32.
Fixes: ca8bc9240a00 ("Add post-2038 timestamp support to e2fsprogs") Signed-off-by: Theodore Ts'o <tytso@mit.edu>
misc: update mke2fs's man page regarding the default inode size
Since a23b50cd ("mke2fs: warn about missing y2038 support when
formatting fresh ext4 fs"), the default inode size is 256 bytes
for all filesystems, including small and floppy, except for the
Hurd since it currently only supports 128-byte inodes.
How timestamps are encoded in inodes and superblocks are different.
Unfortunately, commit ca8bc9240a00 which added post-2038 timestamps
was (a) overwriting adjacent superblock fields and/or attempting
unaligned writes to a 8-bit field from a 32-bit pointer, and (b) using
the incorrect encoding for timestamps stored in inodes. Fix both of
these issues, which were found thanks to UBSAN.
Fixes: ca8bc9240a00 ("Add post-2038 timestamp support to e2fsprogs") Signed-off-by: Theodore Ts'o <tytso@mit.edu>
mke2fs: the -d option can now handle tarball input
If archive.h is available during compilation, enable mke2fs to read a
tarball as input. Since libarchive.so.13 is opened with dlopen,
libarchive is not a hard library dependency of the resulting binary.
In comparison with feeding a directory tree to mke2fs via -d this has
the following advantages:
- no superuser privileges, nor fakeroot, nor unshared user namespaces
are needed to create filesystems with arbitrary ownership information
and special files like device nodes which otherwise require being root
- by reading a tarball from standard input, no temporary files need to
be written out first as mke2fs can be used as part of a shell pipeline
which reduces disk usage and makes the conversion independent of the
underlying file system
A round-trip from tarball to ext4 to tarball yields bit-by-bit identical
results
Signed-off-by: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
Commit ca8bc9240a00 ("Add post-2038 timestamp support to e2fsprogs")
was never built or tested on a 32-bit. It introduced some build
problems when time_t is a 32-bit integer, and it exposed some test
bugs. Fix them.
Fixes: ca8bc9240a00 ("Add post-2038 timestamp support to e2fsprogs") Signed-off-by: Theodore Ts'o <tytso@mit.edu>
projects / thirdparty / e2fsprogs.git / log
