Theodore Ts'o [Tue, 7 Jun 2016 15:47:40 +0000 (11:47 -0400)]
tune2fs,resize2fs: special case HURD only when testing s_creator_os
The s_creator_os field was a mistake, given how Hurd has been
ab(using) certain file system fields. We should skip support for
certain advanced features (64-bit support, metadata checksums) for
file systems created on the Hurd OS only, instead of only supporting
them for file systems created on Linux. This fixes various regression
test failures for FreeBSD.
(The regression tests are probably mostly hopeless for Hurd anyway,
but given the HURD abuse's of various file system fields, the HURD is
going to have to be given second-class treatment in any case, given
what they decided to do with ext2 support, which locks them out of the
more advanced file system features, anyway.)
Theodore Ts'o [Tue, 7 Jun 2016 15:44:54 +0000 (11:44 -0400)]
libext2fs: fix EXT2FS_PRETEND_* on non-Linux systems
The debugging environment variables EXT2FS_PRETEND_* were implemented
in check_mntent_file(), and this function isn't called on all
operating systems. Lift this code up to ext2fs_check_mount_point(),
so that these environment variables (which are used in the regression
test suite) will work on all platforms.
Theodore Ts'o [Tue, 7 Jun 2016 04:16:17 +0000 (00:16 -0400)]
libext2fs: don't use $(LDFLAGS_STATIC) when linking test programs
The tst_* progams aren't intended to be linked 100% stically; they
just link against some of the static libraries. So use $(ALL_LDFLAGS)
and not $(LDFLAGS_STATIC).
Theodore Ts'o [Tue, 7 Jun 2016 03:04:43 +0000 (23:04 -0400)]
libext2fs: fix Direct I/O fallback code so it implements RMW correctly
There is a bug in Unix I/O manager where if an aligned I/O is
required, it does not correctly do the read-modify-write cycle
correctly. Specifically, it was not doing an lseek between the read
and the write calls, so the update was going to block N+1 instead of
block N. Oops.
Fortunately in practice we almost never use this fallback path, so
file systems weren't getting horribly corrupted, because (a) we almost
never use Direct I/O in e2fsprogs, at least not by default, and (b)
when we do the buffers end up being aligned anyway, so it's OK.
We only noticed this because the new Undo I/O manager in e2fsprogs
1.43 was doing unaligned I/O and FreeBSD requires that I/O requests be
aligned even if you are not doing Direct I/O, and the e2undo
regression tests were all failing as a result.
Theodore Ts'o [Mon, 6 Jun 2016 20:49:37 +0000 (16:49 -0400)]
tests,debugfs: ea_set's "-f <value_file" needs to be before the other args
For systems that don't use GNU's getopt(3), options have to be before
all non-option arguments. So change the usage message for debugfs's
ea_set command, and then fix the d_xattr_sorting test.
Theodore Ts'o [Tue, 31 May 2016 05:27:24 +0000 (01:27 -0400)]
debian: adjust how we build hardened binaries to avoid blhc complaints
Avoid false positives for blhc. Also, build the object files for the
static library using -fPIE so they can be linked with a shared
executable linked with -pie.
Theodore Ts'o [Tue, 31 May 2016 05:25:58 +0000 (01:25 -0400)]
lib: build elf shared libraries using -fPIC -shared
Make sure we use "-fPIC -shared" consistently on linker and compiler
command lines when building or linking the shared objects for the ELF
shared library.
Theodore Ts'o [Mon, 30 May 2016 00:48:54 +0000 (20:48 -0400)]
Add configure --disable-tdb which disables e2fsck's scratch_files feature
The scratch_files feature is not really needed except on 32-bit
platforms, since tdb's performance is pretty awful given how we are
using it. Maybe SQLite would be faster, but for 64-bit platforms,
enabling swap works fairly well, especially using the rbtree for the
bitmap abstraction.
We leave tdb for Android since it's unlikely that someone will be
trying to connect petabyte+ sized file systems to a mobile handset.
Theodore Ts'o [Sun, 29 May 2016 21:07:17 +0000 (17:07 -0400)]
Fix "make gcc-wall" for library code
Commit fd1677e80: "Add --enable-hardening which builds e2fsprogs with
security hardening" broken the gcc warnings mode by changing how
CFLAGS were used to build static and shared library objects.
Also shut up GCC's whining about casting the return value from
dlsym(2) to function pointers --- since due to conflicing standards,
there is no way to use dlsym() without causing gcc to whine one way or
another.
Theodore Ts'o [Sun, 29 May 2016 05:05:06 +0000 (01:05 -0400)]
e2fsck: always fix invalid extra field in timestamps if time_t is 32-bits
Systems with 32-bit time_t's can be used past 2038, so we should
always modify dates with extra_time=3. (On systems with 64-bit
time_t's we will fix these dates before the year 2242.)
This was previously addressed by Michael Forney on commit 53904ae543591a
but it seems the test cases from lib/e2p (tst_ostype and tst_feature)
were missed.
Fix them by adding the same $(ALL_LDFLAGS) to those two targets.
Tested by building it on a system that depends on LDFLAGS to produce
working binaries and confirming that `make check` succeeds.
Reported-by: Eric Christopher <echristo@google.com> Signed-off-by: Filipe Brandenburger <filbranden@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 25 May 2016 03:11:32 +0000 (23:11 -0400)]
e2fsck: improve error messages when the journal sb is corrupt
If the journal superblock is corrupt and the user declines to fix it
(or runs e2fsck -n), make sure the error messages are clear and
explain that e2fsck cannot (safely) proceed.
Theodore Ts'o [Wed, 25 May 2016 02:21:49 +0000 (22:21 -0400)]
debian: use e2fsck.static from BUILD-STD
Now that we are no longer trying to use dietlibc, there's no point
creating a separate build tree for e2fsck.static. So just use
e2fsck.static from the standard build tree.
Theodore Ts'o [Tue, 24 May 2016 20:48:25 +0000 (16:48 -0400)]
debian: simplify rules file
Modern versions of debhelper understand how to build udeb, so we can
remove a lot of special case rules in the debain/rules file. Also use
dh_lintian instead of open-coding the rules to copy in the Lintian
overrides files.
Theodore Ts'o [Sun, 22 May 2016 07:20:25 +0000 (03:20 -0400)]
debian: loosen up dependency on debhelper
In order to allow this package to be built on jessie, just depend on
debhelper >= 9.0. We will test to see if dh_update_autotools_config
exists and only use it if it is present.
Theodore Ts'o [Sun, 22 May 2016 03:51:19 +0000 (23:51 -0400)]
Add --enable-hardening which builds e2fsprogs with security hardening
Enable the following security features: stack protection, fortify,
read-only relocation tables, immediate dynamic symbol binding, and
text segment ASLR by enabling position independent executable
(PIE).
Special handling is provided for shared library and statically linked
executables. For all the gory details please see:
Distributions who want to do their own special thing can set CFLAGS,
CFLAGS_SHLIB, CLFAGS_STLIB, LDFLAGS, LDFLAGS_SHLIB and LDFLAGS_STATIC
as appropriate.
Theodore Ts'o [Sun, 22 May 2016 02:10:39 +0000 (22:10 -0400)]
e2fsck: fix project quota support
Use a large_inode so that when e2fsck is fixing a file system with
project quota enabled, the correct project id's quota is adjusted when
a corrupted inode is deleted.
Theodore Ts'o [Sun, 22 May 2016 00:31:59 +0000 (20:31 -0400)]
mke2fs: fix project quota creation
Creating a file system with project quotas can fail if mke2fs is built
using hardening options. This is because quota_compute_usage() used
ext2fs_get_next_inode() instead of ext2fs_get_inode_full(), and a
small inode was passed into quota_data_add, when a large inode needs
to be used. As a result get_dq() would end up dereferencing undefined
space in the stack. Without the hardening options, this would be
zero, so "mke2fs -t ext4 -O project.quota -I 256 test.img" would work
essentially by accident.
Fix this by using ext2fs_get_inode_full() so that a large inode is
available to quota_data_inodes().
Theodore Ts'o [Sat, 21 May 2016 04:15:46 +0000 (00:15 -0400)]
debian: remove unused and deprecated substitution variables
Replace "Source-Version" with "binary:Version" in
e2fsprogs.shlibs.local, and e2fsck-static doesn't have any shared
libraries dependency so remove the ${shlibs:Depends} from the Depends.
Theodore Ts'o [Sat, 21 May 2016 03:43:29 +0000 (23:43 -0400)]
debian: backup/restore the po files which get modified during the build
Before running "make update-gmo", backup the po/*.gmo and po/*.po
files so they can be restored when "make -f debian/rules clean" is
run. This allows the Debian source tree to built and rebuilt.
Theodore Ts'o [Fri, 20 May 2016 17:37:58 +0000 (13:37 -0400)]
debian: create dbgsym packages if debhelper supports them
Starting with Debian Stretch, we now have support for dbgsym packages
which are uploaded to a separate package archive. Since some people
might want to try to backport e2fsprogs 1.43 for Jessie set things up
so that the package can be built on older Debiain systems (where the
legacy dbg packages will be created) as well as newer (as of this
writing, Debian Sid or Stretch).
Theodore Ts'o [Tue, 17 May 2016 05:30:42 +0000 (01:30 -0400)]
mke2fs.conf: don't enable metadata_csum by default
For the 1.43 release, we will not be enabling the metadata_csum
feature by default. We will leave that up to distribution vendors to
decide whether they want to be aggressive about enabling this feature
by default.
Marcus Huewe [Thu, 12 May 2016 19:36:12 +0000 (15:36 -0400)]
libext2fs: add fs_offset entry to the undo header
The fs_offset entry stores the filesystem offset. This allows for an
easy undo, because one does not have to remember/specify the
filesystem offset manually.
The fs_offset entry is implemented as a compatible feature.
Signed-off-by: Marcus Huewe <suse-tux@gmx.de> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Marcus Huewe [Thu, 12 May 2016 19:35:44 +0000 (15:35 -0400)]
libext2fs: fix key extension in undo_write_tdb
Support key extension if the tdb_data_size is an arbitrary integer
multiple of the channel's block size. Before, key extension was only
possible if the tdb_data_size and the channel's block size were
equal.
Note: a key, whose data is the result of a short read, will be
extended if the tdb_data_size and the channel's block size are equal
(that's what the old code did) (if tdb_data_size is an arbitrary
integer multiple (> 1) of the channel's block size, the key might
be extended as well (depending on the keysize)).
Signed-off-by: Marcus Huewe <suse-tux@gmx.de> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Marcus Huewe [Thu, 12 May 2016 19:35:04 +0000 (15:35 -0400)]
libext2fs: fix offset code in undo_write_tdb
The old code has some issues, for example, when backing up fs block 0
(can be reproduced via "mke2fs -z undo -b 1024 -E offset=1024 out 1024"):
* backing_blk_num is set to ULLONG_MAX instead of 0
* data is read from the beginning of the file instead of offset 1024
* data_ptr is set to read_ptr - 1024 ("invalid" address)
Marcus Huewe [Thu, 12 May 2016 19:26:09 +0000 (15:26 -0400)]
tests: testcases for the offset handling in e2undo and undo_io.c
The testcase "u_offset" tests/defines e2undo's new UI and offset
handling code. The testcase "u_mke2fs_opt_offset" documents some
issues with undo_io.c's old offset handling code (e.g., backup of
fs block 0, key extension (tdb_data_size != block_size) etc.).
Signed-off-by: Marcus Huewe <suse-tux@gmx.de> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Marcus Huewe [Thu, 12 May 2016 19:08:46 +0000 (15:08 -0400)]
tests: test "mke2fs -E offset=N" filesystem size calculation
If "mke2fs -E offset=N ..." is called without a specified filesystem size,
the offset should be subtracted from the default filesystem size (see
commit 87d9b2fb). If "mke2fs -E offset=N ..." is called with an
explicitly specified filesystem size, the specified filesystem size
should be used.
Also, call dd with the "iflag=fullblock" option, if it is going to read
from a pipe.
Signed-off-by: Marcus Huewe <suse-tux@gmx.de> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 12 May 2016 03:23:06 +0000 (23:23 -0400)]
libext2fs: sanity check group argument to ext2fs_clear_{block,inode}_uninit()
Avoid a potential out-of-bounds memory access if the group passed to
ext2fs_clear_block_uninit() or ext2fs_clear_inode_uninit() is greater
than the number of groups in the file system. This prevents a failure
in resize2fs when to allocate a block when growing the file system
significantly.
Theodore Ts'o [Wed, 11 May 2016 01:06:15 +0000 (21:06 -0400)]
resize2fs, libext2fs: fix bugs in sparse_super2 support
Fix resize2fs so that the location of the backup superblocks when the
sparse_super2 feature is enabled is properly set when growing the file
system from a single block group to larger file system sizes.
Also fix a bug where the block group summary statistics in some cases
when exapnding a sparse_super2 file system.
Finally, accurately calculate the file system metadata overhead of the
last block group in sparse_super2 file systems.
Theodore Ts'o [Tue, 10 May 2016 20:18:44 +0000 (16:18 -0400)]
util: remove lib/support from the e2fsprogs-libs tar distribution
The lib/support library is only used for internal e2fsprogs programs,
and it won't compile correctly due to the fact that lib/ext2fs isn't
included in the e2fsprogs-libs distribution anyway.
Theodore Ts'o [Mon, 9 May 2016 01:11:18 +0000 (21:11 -0400)]
mke2fs: fix the parsing used for -E quotatype=usrquota:grpquota:prjquota
Commit 2d2d799c7261 tried to use parse_quota_options(), which uses
commas to separate out the quota types. Unfortunately, when parsing
extended options, commands are used to separate different extended
options.
To fix this, I've add a new support function parse_quota_type(), which
allows either commas or colons to used as a separator character, and
which manipulates a bit field to indicate which quota types should be
enabled. Eventually tune2fs should be converted to use
parse_quota_type() as well, thus obsoleting parse_quota_options(), but
that's a more complicated cleanup patch for later.
Fix a lint warning which could the number of blocks to be incorretly
printed if it exceeds 2**32.
Also fix some typos and other minor bugs in the usage message.
Theodore Ts'o [Sun, 1 May 2016 04:57:19 +0000 (00:57 -0400)]
mke2fs: adjust the default file system size by the offset
If the user specifies an offset using -E offset without specifying an
explicit size, the system will use the block device (or file) size as
the default file system size. If we are using the default file system
size, subtract out the offset so the resulting file system will stay
within the block device. Also print a warning that this might not be
what the user wants.
Theodore Ts'o [Sun, 1 May 2016 02:19:59 +0000 (22:19 -0400)]
Use texi2dvi --clean to work around a bug in texi2dvi
If the sources for e2fsprogs are located in a pathname that contains a
tilde character (which can happen when the sources are unpacked using
"apt-get source" when e2fsprogs has a pre-release test version),
texi2dvi will fail due to a bug (Debian bug #822492: "texinfo:
texi2dvi doesn't protect the ~ character in the second pass"). Work
around this bug by using texi2dvi --clean.
e2fsck: don't abort if the journal is corrupted due to checksum errors
If e2fsck_run_ext3_journal() returns an error indicating that a CRC
error was detected, we shouldn't abort, but instead proceed so the
file system can be fixed.
e2fsck: don't try to set a UUID on metadata_csum file systems
If the UUID field is NULL, e2fsck will try to generate and set a new
UUID. This will cause massive problems if the metadata_csum feature
is set, so avoid doing so in that case.