Jan Kara [Tue, 25 Oct 2016 18:08:59 +0000 (14:08 -0400)]
mke2fs: Avoid crashes / infinite loops for absurdly large devices
When a device reports absurdly high size, some arithmetics in mke2fs can
overflow (e.g. number of block descriptors) and we end in an infinite
loop. Fix that by checking and refusing insanely large devices.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libsupport: fix compilation on RHEL 5.x due to missing magic define
RHEL 5.x does have magic.h, but it does not define all expected symbols. In
particular, the NO_CHECK symbols were only added in file 4.20 and RHEL 5.x
is using 4.17.
Add substitute defines to allow continued usage of magic but without the
requested exclude checks.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 24 Oct 2016 18:15:43 +0000 (14:15 -0400)]
mke2fs.conf: add back missing uninit_bg feature flag
Commit 0da058af9f88: "mke2fs.conf: don't enable metadata_csum by
default" removed metadata_csum without adding back the uninit_bg flag
--- which had the net result of causing file systems to take a long
time to mke2fs, since this caused all of the block group metadata to
be initialized.
Theodore Ts'o [Fri, 21 Oct 2016 14:21:54 +0000 (10:21 -0400)]
libext2fs: unix_io: reflect error from read/write calls to caller
If the read(2) or write(2) system calls fail, return the error to the
caller instead of returning "short read" or "short write", which is
just misleading.
libext2fs: force DIO alignment FreeBSD when operating on a block device
FreeBSD (and possibly BSD systems) requires that reads and writes to
block devices must be aligned, even when the O_DIRECT flag is not
specified. Previously this was hard-coded to 512 bytes, but in order
to properly handle Advanced Format HDD's, query the BSD kernel to
determine the proper alignment to use.
libext2fs: fix unaligned, multiblock writes in the unix_io handler
The read-modify-write code for the unaligned fallback code wasn't
working for multi-block writes. This was unmasked by FreeBSD 11-rc2,
since its malloc() is returning unaligned memory regions for large
memory regions.
Avoid crashing on unaligned pointers from corrupted file systems
On platforms that don't permit unaligned pointer dereferences,
corrupted file systems will as used by the regression test suite can
cause e2fsck and debugfs to crash. Avoid those crashes caused by
corrupted file systems. With this commit the full set of regression
test suites will pass on the sparc64 platform.
Fix a typo in the @ifNotGNUmake@ case of MCONFIG.in.
Also allow the FORCE_NATIVE_MAKE to force the use of the non-GNU make
optimized Makefile. The resulting makefile will work with GNU Make
4.2.1, although "make V=1" won't be honored.
Commit 35a4e1b1c5cbb introduced a regression which caused e2fsck on
32-bit systems to think all timestamps were legacy pre-1970
timestamps. Fix the bug.
debian: fix build reproducibility problems in the rules file
The configure file needs to be run using a relative pathname so that
when $(srcdir) gets introduced into binary file, the location of the
build directory doesn't find its way into the binaries.
libext2fs: allow the default journal size to go as large as a gigabyte
Recent research has shown that for a metadata-heavy workload, a 128 MB
is journal be a bottleneck on HDD's, and that the optimal journal size
is proportional to number of unique metadata blocks that can be
modified (and written into the journal) in a 30 second window. One
gigabyte should be sufficient for most workloads, which will be used
for file systems larger than 128 gigabytes.
Andreas Dilger [Wed, 10 Aug 2016 22:21:19 +0000 (18:21 -0400)]
fsck: fix strange logic
llvm warns about the confusingly written comparison:
!strncmp(argv[i+1], "-", 1) == 0) {
misc/fsck.c:1178 col 9: warning: logical not is only applied to
the left hand side of comparison [-Wlogical-not-parentheses]
misc/fsck.c:1178 col 9: note: add parentheses after the '!' to
evaluate the comparison first
misc/fsck.c:1178 col 9: note: add parentheses around left hand
side expression to silence this warning
It makes sense to simplify this to a character comparison rather
than using strncmp() to check only one character.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This new manager is similar to the unix_io_manager except it takes a
file descriptor as first argument instead of a filename.
Some programs may want libext2fs to directly use a fd instead of
letting it opening the file.
The use case for such a io_manager would be to let programs use
a fd even if the filename is unknown:
- the fd comes from a temporary file (O_TMPFILE);
- the fd comes from a unix socket...
Refactoring unix_open() also fix a bug when the IO_DIRECT flag was
specified: ext2fs_get_dio_alignment() was called before the file was
actually opened, resulting in an alignment of 0.
When a ext2_filsys is freed, ext2fs_zero_blocks2() frees the buffer but
does not reset its size.
If this function is later called with a new ext2_filsys, the code
assumes that the buffer is still valid and return a NULL pointer.
Valgrind output:
==188948== Syscall param pwrite64(buf) points to unaddressable byte(s)
==188948== at 0x4E46D03: __pwrite_nocancel (syscall-template.S:81)
==188948== by 0x1254EC: raw_write_blk (unix_io.c:240)
==188948== by 0x124AAB: unix_write_blk64 (unix_io.c:850)
==188948== by 0x122893: ext2fs_zero_blocks2 (mkjournal.c:204)
Theodore Ts'o [Wed, 10 Aug 2016 19:49:35 +0000 (15:49 -0400)]
resize2fs: fix crash when there is an ea block and no blocks to migrate
This fixes a bug introduced in 1.43 by commit fb47b94fffc: "resize2fs:
rewrite extent/dir/ea block checksums when migrating". If there is an
extended attribute block and there are no blocks that need to migrate,
we will crash.
The bug was caused by a botched De Morgan's transformation.
libext2fs: fix infinite loop if copying in an empty directory
In m_minrootdir, on FreeBSD 9.3, try_lseek_copy() fails on an empty file
because errcode is uninitialized and the while() loop never executes,
and the errcode garbage is returned.
Initialize errcode = 0 in try_lseek_copy() to avoid a "fail" result when
there was nothing to copy.
When journal is released, s_sequence is set to j_tail_sequence.
But, currently, even if the recovery process is successfully completed,
the j_tail_sequence and, finally, s_sequence are never changed. By this,
when we repeat doing power-off the device suddenly and executing e2fsck
without full scan before mount, the s_sequence number will never change
and, in a very rare case, newly generated journal logs will be
surprisingly grafted to the old journal logs. In this case, out-of-date
metadata log can be replayed on the filesystem area and the filesystem
can be crashed unintentionally by journal recovery process. Therefore,
we need to update j_tail_sequence after recovery process is successfully
completed in e2fsck.
Youngjin had repeated this test and found the problem. With our test,
the filesystem crash occurred within 4 hours.
Eric Sandeen [Thu, 16 Jun 2016 22:51:04 +0000 (17:51 -0500)]
e2undo: fix endian issues
Two new e2undo issues exist in the latest release on big endian
machines.
From sparse check:
undo_io.c:157:26: warning: invalid assignment: |=
undo_io.c:157:26: left side has type restricted __le32
undo_io.c:157:26: right side has type int
undo_io.c:161:26: warning: invalid assignment: &=
undo_io.c:161:26: left side has type restricted __le32
undo_io.c:161:26: right side has type int
e2undo.c:211:16: warning: cast to restricted __le64
e2undo.c:211:16: warning: cast from restricted blk64_t
e2undo.c:212:16: warning: cast to restricted __le64
e2undo.c:212:16: warning: cast from restricted blk64_t
Addresses-RedHat-Bugzilla: 1344636 Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 7 Jun 2016 17:36:47 +0000 (13:36 -0400)]
Remove the last of behaviour-specific checks on EXT2_OS_LINUX
If there is a feature check, we can just depend on the feature check.
If it is something that can't be checked via a feature flag, then
instead of checking for EXT2_OS_LINUX, we should instead check for
*NOT* EXT2_OS_HURD. since HURD is the special case.
Theodore Ts'o [Tue, 7 Jun 2016 16:46:24 +0000 (12:46 -0400)]
tests: suppress use of libmagic when running the regression test suite
How libmagic will behave is not necessarily stable across different
operating systems of versions of libmagic (or the magic number
database). So suppress it to avoid false test failures.
Theodore Ts'o [Tue, 7 Jun 2016 15:52:25 +0000 (11:52 -0400)]
tests: disable some tests for FreeBSD
Unfortunately, FreeBSD doesn't support sparse files in their tmpfs,
and they generlaly don't mount a tmpfs on /tmp anyway. As a result
certain tests will either OOM kill a FreeBSD (if tmpfs is in use) or
will take forever (if it is not in use).
So let's turn off some tests for FreeBSD (m_hugefile is disabled on
MacOS already, for similar reasons). We need to find a better
solution in the long term, but for now, these tests are guaranteed to
be a disaster on FreeBSD, so suppress them for now.
Theodore Ts'o [Tue, 7 Jun 2016 15:47:40 +0000 (11:47 -0400)]
tune2fs,resize2fs: special case HURD only when testing s_creator_os
The s_creator_os field was a mistake, given how Hurd has been
ab(using) certain file system fields. We should skip support for
certain advanced features (64-bit support, metadata checksums) for
file systems created on the Hurd OS only, instead of only supporting
them for file systems created on Linux. This fixes various regression
test failures for FreeBSD.
(The regression tests are probably mostly hopeless for Hurd anyway,
but given the HURD abuse's of various file system fields, the HURD is
going to have to be given second-class treatment in any case, given
what they decided to do with ext2 support, which locks them out of the
more advanced file system features, anyway.)
Theodore Ts'o [Tue, 7 Jun 2016 15:44:54 +0000 (11:44 -0400)]
libext2fs: fix EXT2FS_PRETEND_* on non-Linux systems
The debugging environment variables EXT2FS_PRETEND_* were implemented
in check_mntent_file(), and this function isn't called on all
operating systems. Lift this code up to ext2fs_check_mount_point(),
so that these environment variables (which are used in the regression
test suite) will work on all platforms.
Theodore Ts'o [Tue, 7 Jun 2016 04:16:17 +0000 (00:16 -0400)]
libext2fs: don't use $(LDFLAGS_STATIC) when linking test programs
The tst_* progams aren't intended to be linked 100% stically; they
just link against some of the static libraries. So use $(ALL_LDFLAGS)
and not $(LDFLAGS_STATIC).
Theodore Ts'o [Tue, 7 Jun 2016 03:04:43 +0000 (23:04 -0400)]
libext2fs: fix Direct I/O fallback code so it implements RMW correctly
There is a bug in Unix I/O manager where if an aligned I/O is
required, it does not correctly do the read-modify-write cycle
correctly. Specifically, it was not doing an lseek between the read
and the write calls, so the update was going to block N+1 instead of
block N. Oops.
Fortunately in practice we almost never use this fallback path, so
file systems weren't getting horribly corrupted, because (a) we almost
never use Direct I/O in e2fsprogs, at least not by default, and (b)
when we do the buffers end up being aligned anyway, so it's OK.
We only noticed this because the new Undo I/O manager in e2fsprogs
1.43 was doing unaligned I/O and FreeBSD requires that I/O requests be
aligned even if you are not doing Direct I/O, and the e2undo
regression tests were all failing as a result.
Theodore Ts'o [Mon, 6 Jun 2016 20:49:37 +0000 (16:49 -0400)]
tests,debugfs: ea_set's "-f <value_file" needs to be before the other args
For systems that don't use GNU's getopt(3), options have to be before
all non-option arguments. So change the usage message for debugfs's
ea_set command, and then fix the d_xattr_sorting test.
Theodore Ts'o [Tue, 31 May 2016 05:27:24 +0000 (01:27 -0400)]
debian: adjust how we build hardened binaries to avoid blhc complaints
Avoid false positives for blhc. Also, build the object files for the
static library using -fPIE so they can be linked with a shared
executable linked with -pie.
Theodore Ts'o [Tue, 31 May 2016 05:25:58 +0000 (01:25 -0400)]
lib: build elf shared libraries using -fPIC -shared
Make sure we use "-fPIC -shared" consistently on linker and compiler
command lines when building or linking the shared objects for the ELF
shared library.
Theodore Ts'o [Mon, 30 May 2016 00:48:54 +0000 (20:48 -0400)]
Add configure --disable-tdb which disables e2fsck's scratch_files feature
The scratch_files feature is not really needed except on 32-bit
platforms, since tdb's performance is pretty awful given how we are
using it. Maybe SQLite would be faster, but for 64-bit platforms,
enabling swap works fairly well, especially using the rbtree for the
bitmap abstraction.
We leave tdb for Android since it's unlikely that someone will be
trying to connect petabyte+ sized file systems to a mobile handset.
Theodore Ts'o [Sun, 29 May 2016 21:07:17 +0000 (17:07 -0400)]
Fix "make gcc-wall" for library code
Commit fd1677e80: "Add --enable-hardening which builds e2fsprogs with
security hardening" broken the gcc warnings mode by changing how
CFLAGS were used to build static and shared library objects.
Also shut up GCC's whining about casting the return value from
dlsym(2) to function pointers --- since due to conflicing standards,
there is no way to use dlsym() without causing gcc to whine one way or
another.
Theodore Ts'o [Sun, 29 May 2016 05:05:06 +0000 (01:05 -0400)]
e2fsck: always fix invalid extra field in timestamps if time_t is 32-bits
Systems with 32-bit time_t's can be used past 2038, so we should
always modify dates with extra_time=3. (On systems with 64-bit
time_t's we will fix these dates before the year 2242.)
This was previously addressed by Michael Forney on commit 53904ae543591a
but it seems the test cases from lib/e2p (tst_ostype and tst_feature)
were missed.
Fix them by adding the same $(ALL_LDFLAGS) to those two targets.
Tested by building it on a system that depends on LDFLAGS to produce
working binaries and confirming that `make check` succeeds.
Reported-by: Eric Christopher <echristo@google.com> Signed-off-by: Filipe Brandenburger <filbranden@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 25 May 2016 03:11:32 +0000 (23:11 -0400)]
e2fsck: improve error messages when the journal sb is corrupt
If the journal superblock is corrupt and the user declines to fix it
(or runs e2fsck -n), make sure the error messages are clear and
explain that e2fsck cannot (safely) proceed.
Theodore Ts'o [Wed, 25 May 2016 02:21:49 +0000 (22:21 -0400)]
debian: use e2fsck.static from BUILD-STD
Now that we are no longer trying to use dietlibc, there's no point
creating a separate build tree for e2fsck.static. So just use
e2fsck.static from the standard build tree.
Theodore Ts'o [Tue, 24 May 2016 20:48:25 +0000 (16:48 -0400)]
debian: simplify rules file
Modern versions of debhelper understand how to build udeb, so we can
remove a lot of special case rules in the debain/rules file. Also use
dh_lintian instead of open-coding the rules to copy in the Lintian
overrides files.
projects / thirdparty / e2fsprogs.git / log
