mke2fs: throttle allocating groups progress as well
Throttle updates for the "Allocating Groups" progress updates to once
a second as well. We now do this throttling in libext2fs, so we don't
have to do this for each of mke2fs's progress updates, and because the
updates from ext2fs_allocate_tables() come from within libext2fs
anyway.
Andreas Dilger [Mon, 10 Sep 2012 09:04:47 +0000 (09:04 +0000)]
tests: kill debugfs on interrupted MMP test
If the f_mmp test is interrupted during its test run, then it can
leave debugfs busy-looping in the background. Since f_mmp is a
relatively long-running test, and is likely to be running during
a parallel test run, this can happen fairly often.
Set a signal trap for the f_mmp test script being killed, so that
the background debugfs command will always be killed by the test.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
mke2fs: disable progress reporting in mke2fs.conf for regression tests
Add a configuration knob so the regression tests can disable progress
reporting. This fixes a potential lack of predictability since the
progress reports are now time based (once a second) which is
problematic for regression tests which are comparing the expected
output of mke2fs.
mke2fs: throttle progress updates to once a second
With lazy itable initialization, the progress updates for writing the
inode table happens so quickly that on a serial console, the time to
write the progress updates can be the bottleneck. Fix this by only
updating the progress indicator once a second.
resize2fs: grow uninit_bg file systems more efficiently
If the uninit_bg feature is enabled and the kernel supports
lazy_itable_init, skip zeroing the inode table so that the resize
operation can go much more quickly. Also set the itable_unused fields
so that the first e2fsck after the resize will run faster.
resize2fs: enforce restrictions if the kernel doesn't do meta_bg resizing
Enhance the online resizing code to be more nuanced about resizing
restrictions. If the kernel supports meta_bg resizing, then we can
skip all of the restrictions. If the kernel does not support meta_bg
resizing, check more carefully to make sure there are enough reserved
gdt blocks, so that the user gets a clearer error message.
Theodore Ts'o [Fri, 31 Aug 2012 19:31:50 +0000 (15:31 -0400)]
resize2fs: allow meta_bg/64-bit file systems to be online resized
Resize2fs can't handle resizing flex_bg file systems that do not have
the resize inode, but when the kernel adds support for resizing using
the meta_bg layout, we should allow it be able to resize the file
system.
So move the flex_bg/resize_inode check to the just before we start
doing the off-line resize, instead of doing it earlier where it would
prohibit these file systems for both on-line and off-line resizes.
resize2fs: fix overhead calculation for meta_bg file systems
The file system overhead calculation in calculate_minimum_resize_size
was incorrect meta_bg file systems. This caused the minimum size to
underflow for very large file systems, which threw resize2fs into a
loop generally lasted longer than the user's patience.
resize2fs: enforce the 16TB limit on 32-bit file systems correctly
The 16TB limit must be enforced regardless of whether the new size is
specified on the command line or implied by the size of the device,
but only if the file system does not support 64-bit block sizes, or
the kernel does not advertise support of meta_bg resizing.
Previously we were unconditionally enforcing it when it was implied by
the device size, but not if the new size was specified on the command
line.
Allow e2fsprogs to be built using the clang (LLVM) frontend
Since clang uses C99 semantics by default, the main changes required
to allow clang to build e2fsprogs was to add support the C99 inline
semantics, while still allowing us to be built when the legacy (but
still default for gcc) GNU C89 inline semantics are in force.
Akira Fujita [Thu, 30 Aug 2012 11:16:01 +0000 (20:16 +0900)]
mke2fs: recalculate the reserved blocks when the last BG is dropped
mke2fs -m option can set reserved blocks ratio up to 50%. But if the
last block group is not big enough to support the necessary data
structures, it gets dropped, we have to recalculate the number of
reserved blocks so that the reserved blocks matches the requested
percentage.
It also avoids a problem where if the user specifies a reserved blocks
of 50%, and after the last partial block group was dropped, if the
number of reserved blocks is greater than 50%, e2fsck will complain.
Steps to reproduce:
1. Create a FS which has the overhead for the last BG
and specify 50 % for reserved blocks ratio
# mke2fs -m 50 -t ext4 DEV 1025M
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
656640 inodes, 2621440 blocks 1310848 blocks (50.00%) reserved for the super user
~~~~~~~ <-- Reserved blocks exceed 50% of FS blocks count!
2. e2fsck outputs filesystem corruption
# e2fsck DEV
e2fsck 1.42.5 (29-Jul-2012)
Corruption found in superblock. (r_blocks_count = 1310848).
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 32768 <device>
The configure option --enable-relative-symlinks was incorrectly
specified in configure.in, as --enable-symlink-relative-symlinks. Fix
the configure script so that --enable-relative-symlinks works, as well
as previous incorrect command line option. We will keep the older,
incorrect --enable-symlink-relative-symlinks for at least two years
before removing it.
Jim Keniston [Mon, 6 Aug 2012 22:46:03 +0000 (18:46 -0400)]
e2fsck: fix potential segv when handling a read error in a superblock
When passed a negative count (indicating a byte count rather than
a block count) e2fsck_handle_read_error() treats the data as a full
block, causing unix_write_blk64() (which can handle negative counts
just fine) to try to write too much. Given a faulty block device,
this resulted in a SEGV when unix_write_blk64() read past the bottom
of the stack copying the data to cache. (check_backup_super_block ->
unix_read_blk64 -> raw_read_blk -> e2fsck_handle_read_error)
Reported-by: Alex Friedman <alexfr@il.ibm.com> Signed-off-by: Jim Keniston <jkenisto@us.ibm.com> Signed-off-by: Dan Streetman <ddstreet@us.ibm.com> Reviewed-by: Mingming Cao <mcao@us.ibm.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 4 Aug 2012 20:56:55 +0000 (16:56 -0400)]
Put ELF_OTHER_LIBS in the right place for the linker
Commit a7c17431b9 attempted to fix a problem where the system
libraries might get used instead of local libraries for things like
-lcom_err. It tried to accomplish this by moving $(ELF_OTHER_LIBS) to
before $(LDFLAGS).
Unfortunately, this was the wrong fix; $(ELF_OTHER_LIBS) *MUST* be
after the object files, or the linker might not pull in the necessary
library and not include it into the DT_NEEDED section of the shared
library. The proper fix is to add a -L$(LIB) before $(LDFLAGS), and
then remove the -L option from all of the ELF_OTHER_LIBS definitions
in the library Makefiles.
e2fsck: add e2fsck.conf options for extra reporting
Add report_time, report_verbose, and report_features options to
e2fsck.conf which enable additional, more verbose reporting by e2fsck.
This is useful for large cloud installations where there are a large
number file systems being managed, and where it may not be obvious
from the e2fsck log files exactly how a particular file system is
configured.
The report_time and report_verbose options, which are the same as the
-tt and -v command line options, respectively, are useful because they
are options specific to e2fsck, and the fsck program does not have a
way of passing certain options only to a specific /sbin/fsck.<fstype>
program.
... will cause mke2fs to write until it fills the device. The cause
for this is that the explicit request for 256 inodes causes the number
of inodes per block group to be 8. The ext2fs_initialize() function
assumed that all of the reserved inodes would be in the first block
group, which is not true in this case. This caused the number of
uninitialized inodes in the first block group to be negative, which
then resulted in mke2fs trying to zero out a very large number of
blocks. Oops.
e2fsck: fix code which uniquifies names in directory entries
When checking to see whether or not a new name is unique, the code was
using the wrong length parameter, which could cause the anti-collision
loop for a long time trying to find what it thinks is a unique name.
Link shared libraries with local libraries instead of system ones
ELF_OTHER_LIBS usually contains local search dirs (-L ../..), but it
was added in link command after system search dirs from LDFLAGS.
Libraries and executables were linked with the system libraries if
present, and possibly using static archives instead of shared
libraries.
It could also make final executable link to fail when shared libraries
are enabled: if libext2fs.so is linked with a static libcom_err.a from
system, build system would attempt to link without -lpthread.
This fixes the issue by moving ELF_OTHER_LIBS before LDFLAGS in the
link command.
If package foo:i386 depends on e2fsprogs, without Multi-Arch: foreign
statetment, the i386 version of e2fsprogs will be installed. By
setting the foreign field, the already installed (in this example
amd64) e2fsprogs package is enough to satisfy the dependency.
The M-A: foreign field is ignored in pre-multiarch systems, so ifdeffing
it using macros should be unnessary.
e2fsck: check a file system mounted read-only if forced
Previously e2fsck would only allow a mounted file system to be checked
if it was the root file system and it was mounted read-only. Now
allow any file system mounted read-only if the -f option is specified.
This makes it easier to test how e2fsck handles checking file systems
which are mounted without having to test on the root file system.
The original reason for this commit was to speed up boots for hard
drives. However, I've measured the time difference on a 1TB laptop
drive, and it's not significant: 70ms vs 10ms when running e2fsck on a
clean file system.
The problem with this optimization is that we don't notice if the
journal superblock has a non-zero s_errno field. If we don't transfer
the error indicator from the journal superblock to the file system
superblock, then the kernel will transfer it when the file system is
remounted read-write, causing scary messages to appear in the syslog.
(And since there was a bug in the kernel code which didn't clear the
error indicator in the journal superblock, it would never get
cleared.)
Reported-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
File is not regular file
"/dev/mapper/testvg-testlv"
due to it being a link, and e4defrag on the link target does as well:
Filesystem is not mounted
due to the target not being found in /etc/mtab.
Fix this by checking whether the symlink target is a block device
and if so, using that device in main(), and also changing get_mount_point()
to search for a matching device number, not device name.
Addresses-Red-Hat-Bugzilla: #707209
Reported-by: Peter Hjalmarsson <xake@rymdraket.net> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Liu Bo [Sat, 28 Jul 2012 21:29:13 +0000 (17:29 -0400)]
filefrag: fix report of number of extents
filefrag has several bugs:
1.
$ touch f1
$ filefrag f1
f1: 1 extent found ----> bug!
$ filefrag -v f1
Filesystem type is: ef53
File size of f1 is 0 (0 blocks, blocksize 4096)
f1: 0 extents found
2.
$ truncate -s 1m f2
$ filefrag f2
f2: 1 extent found ----> bug!
$ filefrag -v f2
Filesystem type is: ef53
File size of f2 is 1048576 (256 blocks, blocksize 4096)
f2: 0 extents found
3.
$ for i in `seq 11 -2 0`; do dd if=/dev/zero of=f4 bs=4k count=1 seek=$i conv=notrunc oflag=sync &>/dev/null; done
$ ll f4
-rw-r--r-- 1 root root 49152 Jun 9 15:09 f4
$ filefrag f4
f4: 7 extents found
$ filefrag -v f4
Filesystem type is: ef53
File size of f4 is 49152 (12 blocks, blocksize 4096)
ext logical physical expected length flags
0 1 1109993 1
1 3 11099921109994 1
2 5 11099911109993 1
3 7 11099901109992 1
4 9 11099891109991 1
5 11 11082071109990 1 eof
f4: 7 extents found -----------------------> but we only have 6 extents, bug!
All of these bugs come from the fact that we've made a mistake on
calculating total extents:
o we set 1 as default for 'total extents', and this will report 1
extent found even when we don't get any extent from fiemap.
o if our first extent does not start from 0(logical addr), total
extents will be one more than what it should be.
Addresses-Red-Hat-Bugzilla: #840848
Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
logsave: fix fd leak which could cause logsave to hang
The logsave program is leaking a file descriptor when it forks and
execs the program which forks a process which hangs around. In the
case of /etc/init.d/checkroot, this would be fsck. This file
descriptor never gets closed, so it's still present when fsck runs
e2fsck, and then if e2fsck has its own logging enabled using (in
/etc/e2fsck.conf):
then e2fsck will fork off a process waiting for /mnt to get remounted
read/write. This causes logsave to never get an EOF from its pipe, so
it hangs waiting for the read to fail --- which won't happen due to
the file descriptor leak which is still being held open by e2fsck's
forked child process. And so /etc/init.d/checkroot hangs, and the
root file system never gets remounted read/write, and we deadlock.
Fix the problem by closing the pipe fd so the logsave program doesn't
end up leaking it to its descendent processes.
Aditya Kali [Fri, 13 Jul 2012 22:25:09 +0000 (15:25 -0700)]
tune2fs: fix quota feature removal
When the last quota inode is removed, the 'quota' feature
flag was not removed from superblock in some cases.
Ex:
$ mke2fs -t ext4 -O quota <dev> # creates both usr & grp
# quota inodes
$ tune2fs -Q ^usrquota <dev> # removes usr quota inode
$ tune2fs -Q ^grpquota <dev> # removes grp quota inode,
# but the 'quota' feature flag
# was not removed from superblock
This patch removes the 'quota' feature flag from superblock
if none of the quota inodes are set.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Aditya Kali [Fri, 13 Jul 2012 22:25:07 +0000 (15:25 -0700)]
libquota: fix quota_inode_truncate()
We failed to clear EXT2_FLAG_SUPER_ONLY after deleting the
quota inode and so, the updated block bitmap was not written
back. This caused fsck to complain after running
'tune2fs -O ^quota <dev>'. Clear this flag so that updated
block bitmap gets written. Also, avoid truncating the quota
inode if it is not hidden.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Aditya Kali [Fri, 13 Jul 2012 22:25:06 +0000 (15:25 -0700)]
tune2fs/quota: always create hidden quota files
Currently 'tune2fs -O quota <dev>' will try to use existing
quota files and write their inode numbers in the superblock.
Next e2fsck run then converts these into hidden quota inodes
(ino #3 & #4). But this approach has problems:
1) Before e2fsck run, the inodes are visible to the user and
might get corrupted or removed or replaced by the user.
2) Since these are user visible, we have to include
their block usage in the quota accounting. But once
these inodes are hidden, e2fsck will have to decrement
their usage from the quota accounting (which e2fsck
currently doesn't do and instead reports error).
(the following used to give e2fsck error previously:
# assume <dev> has aquota.user & aquota.group files
$ tune2fs -O quota <dev> # stores ino# of quota files in
# ext4 superblock
$ e2fsck -f <dev> # hides quota files, but now quota
# usage is incorrect.
<< quota errors >>
Instead of making e2fsck complicated, this patch creates the
hidden quota inodes at 'tune2fs -O quota' time iteself. The
usage is computed freshly and limits are copied from the
aquota.user and aquota.group files as earlier.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 6 Jul 2012 01:44:51 +0000 (19:44 -0600)]
build: add generated files to .gitignore
Add a batch of files/wildcards to the list of ignored files, so
that "git status" does not show so many files that should be ignored.
The "*.dSYM" directories are generated during building on MacOS.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 6 Jul 2012 01:44:20 +0000 (19:44 -0600)]
e2fsck: rename "bool" variables
Since "bool" is a valid C type, declarations of the form "int bool"
will cause compiler errors if <stdbool.h> is included. Rename these
variables to avoid this name clash.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 13 Jun 2012 19:36:56 +0000 (15:36 -0400)]
Update version.h using the proper abbreviation for June
The configure script uses the date string in version.h to calculate a
version date code. This only used for tagging prerelease tarball, so
it's not a big deal, but fix this for correctness' sake.
Theodore Ts'o [Wed, 13 Jun 2012 19:29:13 +0000 (15:29 -0400)]
Fix blhc (Build Log Hardening Check) warnings
The Build Log Hardening Check is a debian tool which scans the output
of a package build making sure that the security hardening flags are
used when compiling and linking all of binaries in a package.
For the most part we were passing CFLAGS, CPPFLAGS, and LDFLAGS down
to the compiler and link commands, but there there were one or two
exceptions. In addition, there where a few places in "make install"
where the V=1 option was not being honored, which triggered blhc
warnings since it couldn't analyze those commands.
The e2fsck.static was the only binary that was not getting built and
packaged with the hardening flags, but I've fixed all of the blhc
warnings so in the future it will be obvious if we regress.
Theodore Ts'o [Tue, 12 Jun 2012 22:25:34 +0000 (18:25 -0400)]
libquota: remove unneeded #include of <sys/quota.h>
The attempted inclusion of sys/quota.h is causing failures in when
building on the hurd and freebsd platforms for Debian. It's not
necessary any more, so just remove the #include.
Theodore Ts'o [Tue, 12 Jun 2012 04:27:39 +0000 (00:27 -0400)]
libext2fs: fix block iterator for extents when truncating inodes
When e2fsck uses the block iterator to release the blocks in an
extent-mapped inode, when the last block in an extent is removed, the
current extent has been removed and the extent cursor is now pointing
at the next inode. But the block iterator code doesn't know that. So
when it tries to go the next extent, it will end up skipping an
extent, and so the inode will be incompletely truncated.
The fix is to go to the next extent before calling the callback
function for the current extent. This way, regardless of whether the
current extent gets removed, the extent cursor is still pointing at
the right place.
Reported-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 12 Jun 2012 02:18:25 +0000 (22:18 -0400)]
e2fsck: update global free blocks/inodes count when truncating orphan inodes
By the time we start processing the orphan inode list, we have already
calculated the total expected number of free blocks and inodes in
ctx->free_{blocks,inodes}. This is used to set the free blocks/inodes
count in the superblock in the case where we don't need to do a full
e2fsck.
We need to update these expected free block counts as we process the
orphan inode list so that superblock values are set correctly.
Otherwise we could have the following happen:
Theodore Ts'o [Tue, 12 Jun 2012 02:02:17 +0000 (22:02 -0400)]
debugfs: interpret date strings of the form @dddd
Debugfs will now interpret date strings of the form @123 as 123
seconds after the start of the epoch. This is handy when editing an
orphan inode linked list using the inode's deletion time field.
Tao Ma [Mon, 11 Jun 2012 03:56:30 +0000 (23:56 -0400)]
tests: add new test f_zero_extent_length
If all of the extents in the last extent tree block (ETB) in a
non-trivial extent tree contain uninitialized extents which are after
the end of the file as defined by i_size, the hueristics will
incorrectly estimate the last entry (and hence the node's e_len field)
in the last entry of each level of the extent tree.
As Tao Ma has noted, since e2fsck was requiring that the length
(e_len) field of interior nodes be non-zero, this was causing false
failures where e2fsck would declare that the extent tree was
corrupted.
This was fixed in commit 9c40d14841: "e2fsck: only check for
zero-length leaf extents". Add a regression test case to ensure that
this issue remains fixed.
Signed-off-by: Tao Ma <boyu.mt@taobao.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 11 Jun 2012 04:25:45 +0000 (00:25 -0400)]
e2fsck: only check for zero-length leaf extents
The on-disk format for interior nodes in the extent tree does not
encode the length of each entry in the interior node; instead, it is
synthesized/simulated by the extent library code in libext2fs.
Unfortunately, this simulation is not perfect; in particular it does
not work for the last extent in the extent tree if there are
uninitialized blocks allocated using fallocate with
FALLOC_FL_KEEP_SIZE, and it leads to e2fsck incorrectly complaining
about an invalid zero-length extent.
We only need to worry about the extent length for the leaves of the
tree, since it is there were we are checking an on-disk value, as
opposed to a software-generated simulation. So restrict the check of
extent length to leaf nodes in the extent tree.
Reported-by: Tao Ma <tm@tao.ma> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Aditya Kali [Mon, 11 Jun 2012 03:52:11 +0000 (23:52 -0400)]
libext2fs: fix i_blocks for extent leaf create/delete with bigalloc
When libext2fs allocates/deletes an extent leaf, the i_blocks
value is incremented/decremented by fs->blocksize / 512. This
is incorrect in case of bigalloc. The correct way here is to
use cluster_size / 512.
The problem is seen if we try to create a large inode using
libext2fs (say using ext2fs_block_iterate3()) on a bigalloc
filesystem. fsck catches this and complains.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 11 Jun 2012 03:35:43 +0000 (23:35 -0400)]
e2fsck: correctly propagate error from journal to superblock
If the file system is mounted read-only after a file system error has
been detected, the fact that an error occurred is written to the
journal. This is important because while the journal is getting
replayed, the error indication in the superblock may very well get
overwritten.
Unfortunately, the code to propagate the error indication from the
journal to superblock was broken because this was being done before
the old file system handle is thrown away and the file system is
re-opened to ensure that no stale data is in the file system handle.
As a result, the error indication in the superblock was never written
out.
To fix this, we need to move the check if the journal's error
indicator has been set after the file system has been freed and
re-open.
Reported-by: Ken Sumrall <ksumrall@google.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Jan Kara [Mon, 4 Jun 2012 16:51:55 +0000 (12:51 -0400)]
libquota: remove unnecessary definitions
Quite some definitions in quota library are not necessary. Remove them.
Also fold quota.h file into quotaio.h since it didn't contain that many
definitions.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Thu, 31 May 2012 23:19:02 +0000 (19:19 -0400)]
e2fsck: handle an already recovered journal with a non-zero s_error field
If a file system was remounted read-only after a file system
corruption is detected, and then that file system is mounted and
unmounted by the kernel, the journal would have been recovered, but
the kernel currently leaves the s_errno field still set. This is
arguably a bug, since it has already propgated the non-zero s_errno
field to the file system superblock, where it will be retained until
e2fsck has been run.
However, e2fsck should handle this case for existing kernel by
checking the journal superblock's s_errno field even if journal
recovery is not required.
Without this commit, e2fsck would not notice anything wrong with the
file system, but a subsequent mount of the file system by the kernel
would mark the file system's superblock as needing checking (since the
journal's s_errno field would still be set), resulting an full e2fsck
run at the next reboot, which would find nothing wrong --- and then
when the file system was mounted, the whole cycle would repeat again.
I had seen reports of this in the past, but it wasn't until recently
that I realized exactly how this had come about, since normally e2fsck
would be run automatically before the file system is mounted again,
thus avoiding this problem. However, a user using a rescue CD who
didn't run e2fsck before mounting the a file system in this condition
could trigger this situation, and unfortunately, with previous
versions of e2fsprogs and the kernel, there would be no way out no
matter what the user tried to do.
Andreas Dilger [Thu, 31 May 2012 20:39:54 +0000 (16:39 -0400)]
e2fsck: allow checking on mounted root fs
Commit 732e26b98e5c79a4298dbe341f43b54b354bb241 added checks to
prevent e2fsck from being run in filesystem-modifying mode against
a mounted or otherwise busy device, due to several bug reports of
users doing this even with the verbose warnings in check_mount().
However, it also prevented e2fsck from checking a mounted root
filesystem, which will prevent the node from booting. Once again
allow e2fsck to run against the mounted root filesystem if it is
also mounted read-only at the time.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Mon, 28 May 2012 22:58:24 +0000 (16:58 -0600)]
tests: use make rules to run tests in parallel
Change the e2fsck/mke2fs/tune2fs/e2image/debugfs regression tests to
be driven by Makefile rules instead of by a script loop. This allows
the tests to be run in parallel like a build and reduces testing time
significantly.
One major change to the tests themselves is to printing the test name,
description, and status together after the test has passed or failed,
to avoid mixing lines from the tests. The other major change is to
use unique temporary filenames for each test, which was mostly handled
already via b4db1e4c7461a50e18c9fd135b9f1ba6f27e4390, but in some
cases temporary files are changed to use $test_name.tmp to avoid any
collision between running tests.
On my old 2-CPU system it reduced the testing time from 160s to 40s.
Much of the savings is from the MMP test delays running in parallel.
It still takes the time of the slowest test, f_mmp_garbage, though
there will be ongoing benefit in the future as more tests are added
since the wallclock time will not increase linearly for each test.
Tests were run with various combinations of "make -j", and "make -j2"
through "make -j44" repeatedly without any test failures.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Mon, 28 May 2012 14:54:08 +0000 (10:54 -0400)]
libext2fs: don't inline ext2fs_open_file() and ext2fs_stat()
The creation of inline wrappers ext2fs_open_file() and ext2fs_stat()
in commit c859cb1de0d624caa0779fb17d1a53766143136e in ext2fs.h caused
difficulties with the use of headers, since the headers for open64()
and stat64() may already be included (and skip the declaration of the
64-bit variants) before ext2fs.h is ever read. There is no real way
to solve the missing prototypes and resulting compiler warnings inside
ext2fs.h.
Since ext2fs_open_file() and ext2fs_stat() are not performance
critical operations, they do not need to be inline functions at all,
and the needed function headers can be handled properly in one file.
Similarly, posix_memalloc() was having difficulties with headers, and
was being defined in ext2fs.h, but it is now only being used by a
single file, so move the required header there.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 28 May 2012 14:38:06 +0000 (10:38 -0400)]
libquota: remove quota_is_on() which was the last user of quotactl()
The quotactl() system call was being used without the use of a
function prototype. On closer examination, it turns out the one user
of that system call was the quota_is_on() function, which is not used
by e2fsprogs at all. Since libquota is an e2fsprogs-internal library,
and not one that we plan to export any time soon, the simplest thing
to do is to simply remove quota_is_on(), which in turn allows us to
remove all of the infrastructure around using the Linux-specific
quotactl() system call.
Andreas Dilger [Thu, 24 May 2012 21:34:56 +0000 (15:34 -0600)]
e2fsck: fix checks done for mounted vs. read-only
Currently, if e2fsck is run without the "-n" flag (i.e. it
might modify the filesystem), there is no guarantee that it will
open the filesystem with the EXCLUSIVE flag (i.e. O_EXCL) to
prevent the block device from being checked (in most cases this
means mounted, but it could also be an MD/LVM member device).
Conversely, if e2fsck is run with "-n" (i.e. read-only), and
/etc/mtab or /proc/mounts does not report the block device as
mounted then e2fsck thinks the filesystem is unmounted. In this
case, e2fsck incorrectly sets the EXCLUSIVE flag, which causes
the check to fail, even though e2fsck is running read-only.
To fix this, do not open with EXCLUSIVE if it is a read-only check,
and always open with EXCLUSIVE if the filesystem might be changed.
This also prevents filesystem mounts while e2fsck is running.
Also refuse allow e2fsck to run at all if the filesystem is BUSY.
The e2fsck check_mount() was checking for MOUNTED, but not BUSY,
and it should refuse to run outright if the block device is BUSY.
The previous MOUNTED heuristics pre-date the O_EXCL reservation
by the kernel, so there could be uncertainty due to stale /etc/mtab
data, but with newer kernels a busy device should never be modified.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Mon, 28 May 2012 02:13:39 +0000 (22:13 -0400)]
libext2fs: fix rbtree backend for extent lengths greater than 2^32
For a completely full filesystem with more than 2^32 blocks, the
rbtree bitmap backend can assemble an extent of used blocks which is
longer than 2^32. If it does, it will overflow ->count, and corrupt
the rbtree for the bitmaps.
Discovered by completely filling a 32T filesystem using fallocate, and
then observing debugfs, dumpe2fs, and e2fsck all behaving badly.
(Note that filling with only 31 x 1T files did not show the problem,
because freespace was fragmented enough that there was no sufficiently
long range of used blocks.)
Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 22 May 2012 01:58:39 +0000 (21:58 -0400)]
e2fsck: fix precedence bug in built-in quota support
The operator precedence bug means that we might pay atteion to
s_grp_quota_inum even if the RO_COMPAT_QUOTA feature flag is clear.
However, fortunately, this is unlikely to happen in practice.
Cc: Aditya Kali <adityakali@google.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 22 May 2012 01:30:45 +0000 (21:30 -0400)]
e2fsck: fix 64-bit journal support
64-bit journal support was broken; we weren't using the high bits from
the journal descriptor blocks! We were also using "unsigned long" for
the journal block numbers, which would be a problem on 32-bit systems.
Theodore Ts'o [Mon, 14 May 2012 16:45:54 +0000 (12:45 -0400)]
Change pkg-config files so that both <et/com_err.h> and <com_err.h> work
Change the include path in the Cflags field so that #include
<lib/foo.h> and <foo.h> will work. We had originally used a C flags
which allowed <foo.h> to work, but many applications (especially those
not using pkg-config) had been using the <lob/foo.h> formulation which
didn't require an explicit -I{$includedir} option to the C compiler.
If those applications then converted over to pkg-config, and the
e2fsprogs libraries were installed with a prefix other than /usr, so
that the header files were in some directory such as
/usr/local/include, a program that used #include <lib/foo.h> would
fail to compile.
So change the pkg-config files to include both -I{$includedir} and
-I{$includir}/lib.
Theodore Ts'o [Mon, 14 May 2012 14:55:09 +0000 (10:55 -0400)]
libext2fs: fix bug in unix_io corrupted > 16TB file systems on 32-bit systems
The code was assuming that "unsigned long" was 64-bit, which of course
it isn't on 32-bit systems. This caused blocks to get written to the
wrong place.
Theodore Ts'o [Wed, 2 May 2012 18:21:28 +0000 (14:21 -0400)]
Support systems without posix_memalign() and memalign()
MacOS 10.5 doesn't have posix_memalign() nor memalign(), but it does
have valloc(). The Android SDK would like to be built on MacOS 10.5,
so I've added support for a good-enough emulation of memalign()'s
functionality using valloc(), with an explicit test to make sure
valloc() is returning a pointer which is sufficiently aligned given
the requested alignment. This won't work if you try to operate on a
file system with a 16k blocksize using an e2fsprogs built on MacOS
10.5 system, but it is good enough for the common case of 4k
blocksize file systems, and we will let the memory allocation fail in
the alignment is not good enough.
I've also added a unit test for ext2fs_get_memalign() so we can be
sure it's working as expected. I've tested the code paths with
HAVE_POSIX_MEMALIGN defined, HAVE_POSIX_MEMALIGN undefined, and
HAVE_POSIX_MEMALIGN and HAVE_MEMALIGN undefined on an x86 Linux
system, and so I know the valloc() code path works OK. The simplistic
(and less safe) patch at:
Theodore Ts'o [Mon, 7 May 2012 18:41:49 +0000 (14:41 -0400)]
libext2fs: factor out I/O buffer allocation
Create a new function, io_channel_alloc_buf() which allocates I/O
buffers with appropriate alignment if we are using direct I/O. The
original code was sometimes using a larger alignment factor than
necessary, and would always request an aligned memory buffer even when
it was not necessary since the block device was not opened with
O_DIRECT.
Theodore Ts'o [Mon, 7 May 2012 17:25:44 +0000 (13:25 -0400)]
libext2fs: refactor Direct I/O alignment requirement calculations
Create a new function, ext2fs_get_dio_alignment(), which returns the
alignment requirements for direct I/O. This way we can factor out the
code from MMP and the Unix I/O manager. The two modules weren't
consistently calculating the alignment factors, and in particular MMP
would sometimes use a larger alignment factor than was strictly
necessary.
Theodore Ts'o [Mon, 7 May 2012 16:56:07 +0000 (12:56 -0400)]
libext2fs: move the alignment field from unix_io to the io_manager
The align field which indicated the required data alignment of data
buffers was stored in a field specific to the unix_io manager. Move
it to the top-level io_channel structure so it can be better
generalized.
Theodore Ts'o [Wed, 2 May 2012 20:28:48 +0000 (16:28 -0400)]
tests: add new tests f_jnl_32bit and f_jnl_64bit
Add regression tests which make sure e2fsprogs understands the current
32-bit and 64-bit journal format. If a patch breaks the on-disk
format, these tests should warn us about that.
Andreas Dilger [Tue, 24 Apr 2012 20:22:48 +0000 (16:22 -0400)]
e2fsck: quiet harmless inode/blocks errors
Don't consider only an error in the superblock summary as incorrect.
The kernel does not update this field except at unmount time, so
don't print errors during a "-n" run if there is nothing else wrong.
Any other unfixed errors will themselves mark the filesystem invalid.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Aditya Kali [Tue, 24 Apr 2012 18:46:08 +0000 (14:46 -0400)]
e2fsck,libquota: Update quota only if its inconsistent
Currently fsck recomputes quotas and overwrites quota files
whenever its run. This causes unnecessary modification of
filesystem even when quotas were never inconsistent. We also
lose the limits information because of this. With this patch,
e2fsck compares the computed quotas to the on-disk quotas
(while updating the in-memory limits) and writes out the
quota inode only if it is inconsistent.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Aditya Kali [Tue, 24 Apr 2012 18:36:42 +0000 (14:36 -0400)]
e2fsck: Fix check for hidden quota files
Currently e2fsck always incorrectly detects that quota inodes
need to be hidden (even if they are already hidden) and
modifies the superblock unnecessarily. This patch fixes the
check for hidden quota files and avoids modifying the
filesystem if quota inodes are already hidden.
Also, zero-out the old quota inode so that next fsck scan
doesn't complain.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
projects / thirdparty / e2fsprogs.git / log
