Darrick J. Wong [Sat, 11 Jan 2014 19:04:48 +0000 (14:04 -0500)]
libext2fs: mark group data blocks when loading block bitmap
The kernel[1] and e2fsck[2] both react to a BLOCK_UNINIT group by
calculating the block bitmap that's needed to show all the group
blocks for that group (if any) and using that. However, when reading
bitmaps from disk, libext2fs simply imports a block of zeroes into the
bitmap, without bothering to check for group blocks. This erroneous
behavior results in the filesystem having a block bitmap that does not
accurately reflect disk contents, and worse yet makes it seem as
though superblocks, group descriptors, bitmaps, and inode tables are
"free" space on disk.
So, fix the block bitmap loading routines to calculate the correct
block bitmap for all groups and load it into the main fs block bitmap.
This also fixes bogus debugfs output such as:
Group 1: (Blocks 8193-16384) [INODE_UNINIT, BLOCK_UNINIT]
Checksum 0x1310, unused inodes 512
Backup superblock at 8193, Group descriptors at 8194-8217
Reserved GDT blocks at 8218-8473
Block bitmap at 283 (bg #0 + 282), Inode bitmap at 299 (bg #0 + 298)
Inode table at 442-569 (bg #0 + 441)
7911 free blocks, 512 free inodes, 0 directories, 512 unused inodes
Free blocks: 8193-16384
Free inodes: 513-1024
Notice how the "free blocks" range includes the backup sb & GDT area
and doesn't match the free block count.
Worse yet, debugfs' testb command will report those group descriptor
blocks as not being in use unless the user also instructs debugfs to
find a free block first. That is a rather surprising result:
debugfs: testb 8194
Block 8194 not in use
debugfs: ffb 1 16380
Free blocks found: 16380
debugfs: testb 8194
Block 8194 marked in use
Also, remove the part of check_block_uninit() that "fixes" the bitmap
since we're doing that at bitmap load time now.
Darrick J. Wong [Sat, 11 Jan 2014 18:58:15 +0000 (13:58 -0500)]
libext2fs: don't always read backup group descriptors on a 1k-block meta_bg fs
On a filesystem with 1K blocks and meta_bg enabled, opening a
filesystem with automatic superblock detection tries to compensate for
the fact that the superblock lives in block 1. However, the method by
which this is done is later misinterpreted to mean "read the backup
group descriptors", which is not what we want in this case.
Therefore, in ext2fs_open3() separate the 'group zero' adjustment into
its own variable so that we don't get fed backup group descriptors
when we try to load meta_bg group descriptors.
Furthermore, enhance ext2fs_descriptor_block_loc2() to perform its own
group zero correction. The other caller of this function neglects to
do any group-zero correction of their own, so this fixes them too.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 11 Jan 2014 18:54:57 +0000 (13:54 -0500)]
libext2fs: detect correct superblock adjustments when loading backup groups
If ext2fs_descriptor_block_loc2() is called with a meta_bg filesystem
and group_block is not the normal value, the function will return the
location of the backup group descriptor block in the next block group.
Unfortunately, it fails to account for the possibility that the backup
group contains a backup superblock but the regular superblock does
not. This is the case with block groups 48-49 on a meta_bg fs with 1k
blocks; in this case, libext2fs will fail to open the filesystem.
Therefore, teach the function to adjust for superblocks in the backup
group, if necessary.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 9 Jan 2014 20:59:29 +0000 (15:59 -0500)]
util: fix make dependencies for subst
Now that subst is using config.h, we need to fix its dependencies so
that if config.h is missing (i.e., after a "make clean" in the tree)
that it is rebuilt by the Makefile rule.
Theodore Ts'o [Thu, 9 Jan 2014 03:25:04 +0000 (22:25 -0500)]
Fix up workarounds for dietlibc breakage
The dietlibc doesn't support the TZ environment variable, which is
required by the standard. Work around this so that we can run the
regression test suite when building with dietlibc. (This is useful
for finding problems.)
With this change, the only thing which doesn't work as far as dietlibc
is concerned is the posix_memalign test, and the MMP support tests
(because posix_memalign isn't provided by dietlibc, sigh.)
Theodore Ts'o [Sun, 5 Jan 2014 06:09:07 +0000 (01:09 -0500)]
debian: renable use of internal uuid and blkid libraries for e2fsck.static
Now that the diet libc support is working again, we need to use the
internal uuid and blkid library in order for e2fsck.static to be
compiled properly.
Theodore Ts'o [Sun, 5 Jan 2014 05:15:54 +0000 (00:15 -0500)]
Use consistent e2fsprogs-standard types
Some different types such as u_int16_t and __uint32_t have snuck into
e2fsprogs. These types are not guaranteed by any standard, and they
are not provided by dietlibc. Convert them to __u16, __u32,
etc. since these are guaranteed to be provided by e2fsprogs' build.
Theodore Ts'o [Sun, 5 Jan 2014 01:44:29 +0000 (20:44 -0500)]
configure: fix --with-diet-libc
Newer versions of autoconf pull in AC_PROG_GCC as part of
AC_CANONICAL_HOST. So we need check for WITH_DIET_LIBC earlier in
configure.in.
Also, e2fsprogs now needs functions which are found in diet libc's
compat library. So add support for autoconf's LIBS function, and
automatically set libs to include -lcompat.
Finally, disable compiling e4defrag by deault if --with-diet-libc is
specified because the program has too many glibc dependencies.
Baruch Siach [Thu, 2 Jan 2014 18:05:37 +0000 (13:05 -0500)]
e4defrag: choose the best available posix_fadvise variant
Use posix_fadvise64() when available. This allows 64bit offsets on
32bit systems.
[ Modified by tytso to try to use fadvise64() as well, and to remove
the attempt to call the syscall directly, since because and
complexities caused by required dummy arguments on some
architectures, it's not worth the hair. ]
Theodore Ts'o [Fri, 3 Jan 2014 14:39:33 +0000 (09:39 -0500)]
debugfs: clean up read_journal_block() to avoid Coverity complaint
The read_journal_block() function was needlessly complicated, which
made it harder to read/maintain, and it also tripped up Coverity.
Cleaning it up also avoided some signed/unsigned casts, and allows us
to avoid passing got back to the caller, since it wasn't needed.
Theodore Ts'o [Fri, 3 Jan 2014 05:26:43 +0000 (00:26 -0500)]
subst: clean up various coverity nits
Add appropriate error checking for all error returns, and only open
each file that we need to manipulate once, to avoid potential
time-of-check/time-of-use races. (Not that this is likely for this
program, but the result is much more clean.)
We also preserve the atime in the case where the file has not changed.
Theodore Ts'o [Wed, 1 Jan 2014 04:02:44 +0000 (23:02 -0500)]
e2fsck: improve the "superblock corrupt" message
Previously, this message used 8193 as the example alternate
superblock. But for most file systems, the backup superblock is
located at 32768 (since most file systems have a block size of 4k, and
not 1k).
Theodore Ts'o [Tue, 31 Dec 2013 03:41:48 +0000 (22:41 -0500)]
update config.h.in file to enable loopback mount detection
Commit fbabd5c44c23 added loopback mount detection. However, we
failed to update the config.h file, so the code wasn't actually
enabled. Fix this oversight.
Theodore Ts'o [Mon, 30 Dec 2013 22:35:58 +0000 (17:35 -0500)]
debian: rationalize configure options management
As part of the debian build, the e2fsprogs sources are built multiple
times --- for the udeb packages, for e2fsck.static, etc. Make sure
that when we disable the use of the built-in blkid and uuid libraries
(in favor of the ones shipped with util-linux), that we do so for all
of the builds.
Previously we were using the internal blkid library for e2fsck.static.
While this isn't completely broken, the internal blkid library hasn't
been maintained in quite some time, so it's better to use the
util-linux blkid library instead.
Eric Whitney [Mon, 30 Dec 2013 21:43:46 +0000 (16:43 -0500)]
debugfs: restore and tweak original error messaging
In response to reviewer comments, commit fe56188b07 included changes
that modified some of the code used to output error messages when
checking user-supplied block numbers. These changes converted calls
to parse_ulonglong() to calls to strtoblk(). Because strtoblk() calls
parse_ulonglong(), and both output error messages, two redundant and
relatively generic messages were output on each error.
Fix this by removing the error message output from strtoblk(), and
extending it to accept an optional error message argument that it
supplies in lieu of a default to parse_ulonglong(). Also, revert to
the more descriptive original error messages with mods per reviewer
comments, and fix an error message in do_replace_node().
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 30 Dec 2013 20:48:06 +0000 (15:48 -0500)]
debian: use dh_strip --dbg-package to populate the dbg packages
Use the --dbg-package option instead of manually moving the files into
the debug packages. This is simpler to maintain, and avoids a problem
where creating directories using mkdir -p can create world-writeable
/usr/lib directory dependind on the umask used by the Debian buildd.
Theodore Ts'o [Sat, 28 Dec 2013 20:40:56 +0000 (15:40 -0500)]
Use makeinfo --html instead of texi2html
The texi2html utility from the texi2html ceased being developed
upstream in 2011, and upstream has declared it superseded by the
makeinfo utility from the texinfo package.
Theodore Ts'o [Fri, 27 Dec 2013 06:04:32 +0000 (01:04 -0500)]
e2image: write progress information to stderr
This allows "e2image -rp /dev/sdc1 - | bzip2 > sdc1.img.bz2" to work
correctly, so the progress information doesn't corrupt the image being
sent to stdout.
Also add a diagnostic indicating that the -p option is currently only
implemented for raw mode.
Theodore Ts'o [Thu, 26 Dec 2013 03:53:00 +0000 (22:53 -0500)]
dumpe2fs: don't try to print the journal info when using image files
If dumpe2fs tries to print journal info when using an image file
created using e2image, it will crash since the journal isn't
available. So don't even try to print it.
Theodore Ts'o [Wed, 25 Dec 2013 21:33:52 +0000 (16:33 -0500)]
e2image: only skip writing zero blocks when writing to a new file
The e2image progam was originally intended to create image files.
However, some people have started using e2image to copy a file system
from one block device to another, since it is more efficient than
using dd because it only copies the blocks which are in use. If we
are doing this, however, we must not skip writing blocks which are all
zero in the source device, since they may not be zero in the
destination device.
Theodore Ts'o [Wed, 25 Dec 2013 19:20:22 +0000 (14:20 -0500)]
e2image: refactor calls to ext2fs_llseek()
Refactor calls to ext2fs_llseek(). This makes the code more compact,
easier to read, and it also fixes a number of places where we call
ext2fs_llseek() without doing proper error checking.
Phillip Susi [Wed, 25 Dec 2013 04:41:34 +0000 (23:41 -0500)]
e2image: handle SIGINT safely
When doing an in place move, interrupting it past the
point of no return will destroy the filesystem since
parts of it have been overwritten. Catch SIGINT the
first time and issue a warning if this is the case.
Phillip Susi [Mon, 16 Dec 2013 16:32:39 +0000 (11:32 -0500)]
e2image: add offset switches
Add -o and -O switches to specify the offset where the source
and destination filesystems start. This is useful if you have
an image of a partitioned disk or wish to create one.
Theodore Ts'o [Wed, 25 Dec 2013 04:19:07 +0000 (23:19 -0500)]
libext2fs: checksum bg descriptors larger than 64 bytes
Currently the ext4 block group descriptor is 64 bytes. In case we
need to support larger block group descriptors in the future, teach
ext2fs_group_desc_csum() to checksum parts of the block group
descriptors that libext2fs doesn't yet understand.
Theodore Ts'o [Wed, 25 Dec 2013 03:44:14 +0000 (22:44 -0500)]
tests: use the in-tree binaries in the test f_extent_oobounds
Fix the f_extent_oobounds test so that it uses binaries built in the
tree, instead of the binaries in the system PATH (which might not
exist in a chroot environment) when creating the test image.
Theodore Ts'o [Wed, 25 Dec 2013 03:40:01 +0000 (22:40 -0500)]
debugfs: fix extent header in do_write for big-endian machines
Commit a17e9f30 set up the extent header for a new file.
Unfortunately it didn't correctly handle byte swapping; fix this so
the regression tests pass on PowerPC systems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Cc: Eric Sandeen <sandeen@redhat.com>
Eric Whitney [Tue, 24 Dec 2013 16:04:42 +0000 (11:04 -0500)]
debugfs: silence build warning and remove incorrect error message
Commit fe56188b07 modified the code used to check the block number
argument to the command line -s switch, adding a call to com_err().
This causes a compile time warning because the third argument to
com_err() isn't a string. Also, when the block number argument is
bad it outputs an incorrect error message - "Operation not permitted".
Fix these minor problems by removing the call to com_err(). Other
code provides enough error reporting information in this case.
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Eric Whitney [Tue, 24 Dec 2013 06:58:09 +0000 (01:58 -0500)]
e2fsck: fix printf conversion specs in ea_refcount.c
Commit 130e961a6fb9d38997c83c6397cf9ddb5ed2491f changed the type
used to represent block numbers in ea_refcount.c from blk_t to blk64_t
to add support for 64 bit extended attribute refcounting. We also
need to adjust printf conversion specs that now don't match their new
blk64_t arguments. This will silence compiler warnings seen when
"make check" is run and will avoid truncation of printed values.
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 23 Dec 2013 21:09:49 +0000 (16:09 -0500)]
mke2fs: further updates for mke2fs(8) man page
The mke2fs(8) man page was updated in 4727c67dc2, but needs some
more clear descriptions for extra_isize and metadta_csum features.
The uninit_bg feature is supported by all ext4-capable kernels,
and does have a slow e2fsck pass for newly-formatted filesystems,
so remove the caveat.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 23 Dec 2013 21:04:46 +0000 (16:04 -0500)]
debugfs, e2fsck: fix s_desc_size handling
The s_desc_size in the superblock specifies the group descriptor
size in bytes, but in various places the EXT4_FEATURE_INCOMPAT_64BIT
flag implies that the descriptor size is EXT2_MIN_DESC_SIZE_64BIT
(64 bytes) instead of checking the actual size. In other places,
the s_desc_size field is used without checking for INCOMPAT_64BIT.
In the case of ext2fs_group_desc() the s_desc_size was being ignored,
and assumed to be sizeof(struct ext4_group_desc), which would result
in garbage for any but the first group descriptor. Similarly, in
ext2fs_group_desc_csum() and print_csum() they assumed that the
maximum group descriptor size was sizeof(struct ext4_group_desc).
Fix these functions to use the actual superblock s_desc_size if
INCOMPAT_64BIT.
Conversely, in ext2fs_swap_group_desc2() s_desc_size was used
without checking for INCOMPAT_64BIT being set.
The e2fsprogs behaviour is different than that of the kernel,
which always checks INCOMPAT_64BIT, and only uses s_desc_size to
determine the offset of group descriptors and what range of bytes
to checksum.
Allow specifying the s_desc_size field at mke2fs time with the
"-E desc_size=NNN" option. Allow a power-of-two s_desc_size
value up to s_blocksize if INCOMPAT_64BIT is specified. This
is not expected to be used by regular users at this time, so it
is not currently documented in the mke2fs usage or man page.
Add m_desc_size_128, f_desc_size_128, and f_desc_bad test cases to
verify mke2fs and e2fsck handling of larger group descriptor sizes.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 23 Dec 2013 21:03:41 +0000 (16:03 -0500)]
e2fsck: verify s_desc_size is power-of-two value
Add a LOG2_CHECK mode for check_super_value() so that it is easy
to verify values that are supposed to be power-of-two values
(s_desc_size and s_inode_size so far). In ext2fs_check_desc()
also check for a power-of-two s_desc_size.
Print out s_desc_size in debugfs "stats" and dumpe2fs output, if
it is non-zero.
It turns out that the s_desc_size validation in check_super_block()
is not currently used by e2fsck, because the group descriptors are
verified earlier by ext2fs_check_desc(), and even without an
explicit check of s_desc_size the group descriptors fail to align
correctly on disk. It makes sense to keep the check_super_block()
regardless, in case the code changes at some point in the future.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 23 Dec 2013 21:01:51 +0000 (16:01 -0500)]
tests: delete temporary resize test files
Since commit 5ad07acad if $TMP cannot hold large test filesystems
for resize testing the resize_test creates temporary test files
in the local working directory. Since it overrides TMPFILE locally
the calling program does not delete the generated file correctly.
Delete the large $TMPFILE within resize_test if it passes, but leave
it for debugging if the test failed.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
projects / thirdparty / e2fsprogs.git / log
