Peng Tao [Thu, 2 Jul 2009 04:24:15 +0000 (00:24 -0400)]
filefrag: fix fm_start in filefrag_fiemap loop
When used with -v and the targeted file has more than 144
extents(double of the length of fm_extents array provided by buf),
filefrag_fiemap loops and calls fiemap ioctl() multiple times to
calculate the actual number of extents in a file. Each call to fiemap
ioctl() uses fm_start as the starting logical offset. The patch fixes
fm_start in each loop( except for the first one) and makes the extent
calculation correct for files with more that 144 extents.
To produce the problem, first run filefrag -v on a highly fragmented
file. Then change the buf size in filefrag_fiemap to make it large
enough to have all the extent mapped in a single loop and run filefrag
-v after recompiling. The former will produce a much smaller extent
count because of the false fm_start used in the loop. And the two will
produce different extent output since the 145th extent.
Signed-off-by: Peng Tao <bergwolf@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add support for configure --enable-verbose-makecmds
Some people don't want to see the concise "kernel-style" make output.
This configure option allows build engines that want to see the full
set of commands executed by the makefile to get what they want. Most
people will find this more distracting than useful, unless they need
to debug the Makefiles.
(It is not necessary to rerun configure to enable this verbose make
output temprarily; if a developer wants to do a quick debug of a
directory's makefile, he or she can simply edit the definition of the
$(E) and $(Q) variables in the Makefile; instructions can be found in
the MCONFIG file which is included in at the beginning of every
Makefile.)
The e2fsprogs makefiles were using the same Makefile variable
LIBCOM_ERR for the link-line arguments as well as the dependencies.
Since LIBCOM_ERR can now include non-file arguments such as
"-lpthread", we need to use a separate DEPLIBCOM_ERR variable that
only has build file dependencies.
Do the same thing for STATIC_LIBCOM_ERR and PROFILED_LIBCOM_ERR.
libuuid: Don't run uuidd if it would fail due to permission problems
Some distributions don't like installing uuidd setuid or setgid. So
if the setuid or setigid bit is not set with uuidd, and the current
process does not have write access to the UUIDD work directory, don't
try running uuidd, since it won't work properly.
Theodore Ts'o [Tue, 30 Jun 2009 00:03:20 +0000 (20:03 -0400)]
libuuid, uuidd: Avoid infinite loop while reading from the socket fd
If for some reason the uuidd daemon or the process calling uuidd
exited unexpectely, the read_all() function would end up looping
forever, either in uuidd or in libuuid. Fix this terminating the loop
if no data can be read after five tries to read from the file
descriptor.
Theodore Ts'o [Mon, 29 Jun 2009 23:32:50 +0000 (19:32 -0400)]
uuidd: Avoid closing the server socket when calling create_daemon()
In the event that file descriptors 0-2 are closed when uuidd is
started, the server socket could be created as a file descriptor that
will get closed when create_daemon() tries detaching the uuidd daemon
from its controlling tty. Avoid this case by using dup(2).
Theodore Ts'o [Mon, 29 Jun 2009 23:06:45 +0000 (19:06 -0400)]
libuuid: Make sure fd's 0, 1, and 2 are valid before exec'ing uuidd
When closing all of the file descriptors before starting uuidd, make
sure file descriptors 0, 1, and 2 are reserved by opening /dev/null.
This prevents strange bugs caused by assumptions regarding file
descriptors <= 2 as being special.
Theodore Ts'o [Mon, 29 Jun 2009 18:58:07 +0000 (14:58 -0400)]
logsave: Don't send the ^A and ^B delimiters to the console
Some terminal programs may print wierd characters when they see the
\001 or \002 characters. So filter them out if the -s option
(skip_mode) is enabled.
ext2fs_validate_entry would read beyond the end of the block to get
dirent->rec_len for certain arguments (like if blocksize ==
final_offset). This patch adds a check so that doesn't happen, and
changes the types of the arguments to avoid a compiler warning.
Signed-off-by: Nic Case <number9652@yahoo.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add an option to switch between the private (in-tree) libuuid and
public (in-system installed) library. The private version is still
enabled by default.
Signed-off-by: Scott James Remnant <scott@netsplit.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Thu, 18 Jun 2009 22:51:07 +0000 (17:51 -0500)]
lsattr: exit with a non-zero status on errors
lsattr doesn't return an error if you point it at a file that
doesn't exist.
This is slightly trickier because it can take more than one
file as an arg, but ls seems to report an error if any occurred,
so this does the same, it'll report the last error that was
encountered.
Addresses-RedHat-Bugzilla: #489841 Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 22 Jun 2009 01:07:38 +0000 (21:07 -0400)]
Fix encoding for rec_len in directories for >= 64k blocksize file systems
Previously e2fsprogs interpreted 0 for a rec_len of 65536 (which could
occur if the directory block is completely empty in 64k blocksize
filesystems), while the kernel interpreted 65535 to mean 65536. The
kernel will accept both to mean 65536, and encodes 65535 to be 65536.
This commit changes e2fsprogs to match.
We add the encoding agreed upon for 128k and 256k filesystems, but we
don't enable support for these larger block sizes, since they haven't
been fully tested.
Eric Sandeen [Thu, 18 Jun 2009 01:55:39 +0000 (20:55 -0500)]
resize2fs: update sb journal backup if journal was moved
This was reported in Fedora, since the livecd creator does
a lot of resizing.
If we've moved the journal blocks during resize (more likely now,
due to the journal being in the middle) the backup blocks in the
superblock don't get updated, and a subsequent e2fsck will find
issues:
e2fsck 1.41.6 (30-May-2009)
Backing up journal inode block information.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/mnt/test/img: ***** FILE SYSTEM WAS MODIFIED *****
/mnt/test/img: 11/16000 files (0.0% non-contiguous), 17789/38400 blocks
Eric Sandeen [Wed, 17 Jun 2009 02:50:53 +0000 (21:50 -0500)]
libext2fs: initialize large inodes for journal & resize
I noticed that neither the journal nor resize inodes have
i_extra_isize set post-mkfs; while this isn't likely
to be a big problem, I think the below patch tidies
it up.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Wed, 17 Jun 2009 22:49:01 +0000 (18:49 -0400)]
libext2fs: write only core inode in update_path()
The ext2_extent_handle only has a struct ext2_inode allocated on
it, and the same amount copied into it in that same function,
but in update_path() we're possibly writing out more than that -
for example 256 bytes, from that address. This causes uninitialized
memory to get written to disk, overwriting the parts of the
inode past the osd2 member (the end of the smaller structure).
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 15 Jun 2009 05:34:43 +0000 (01:34 -0400)]
libext2fs: Fix memory leak in the extents handling function
Each time an extent handle is opened and closed, if the inode has an
extent tree which does not fit in the inode's i_block structure, a
filesystem block buffer was not getting released. Since e2fsck opens
an extent handle for every inode using extents, this can translate to
a very large amount of memory getting lost.
Thanks to Henrik 'Mauritz' Johnson for discovering and pointing out
this leak, which he ran into while running the "rdump" command in
debugfs.
Theodore Ts'o [Sat, 30 May 2009 16:34:28 +0000 (12:34 -0400)]
Update man pages to include ext4 in the synposis, instead of just "ext2/ext3"
The e2fsprogs programs have historically just said that they operate
on ext2 and ext3 file system in their man pages. Update them to say
that they also operate on ext4 file systems.
Theodore Ts'o [Fri, 29 May 2009 15:01:22 +0000 (11:01 -0400)]
libext2fs: Move test code for csum.c inline instead of in a separate file
Move the test code which is used to build the tst_csum progam from
tst_csum into csum.c under an #ifdef DEBUG to simplify things and to
avoid compile problems caused by not having a prototype for
ext2fs_group_desc_csum().
Theodore Ts'o [Fri, 29 May 2009 04:09:57 +0000 (00:09 -0400)]
libext2fs: Add an explicit error code for missing mtab file
To reduce user confusion, if the /etc/mtab file is missing
ext2fs_check_mount_point and ext2fs_check_if_mounted will return a
new, explicit error code to indicate this case.
Theodore Ts'o [Fri, 29 May 2009 03:40:18 +0000 (23:40 -0400)]
e2fsck: Go back to the original superblock if the backup sb is invalid
In the case where the block group descriptors appear corrupt, e2fsck
will try to use the backup superblock. However, it could be that the
backup superblock itself is completely corrupted, in which e2fsck
should go back to the original superblock instead of refusing to fix
the file system.
Theodore Ts'o [Fri, 29 May 2009 02:03:33 +0000 (22:03 -0400)]
debugfs: Add -a option to close_filesys command
Add an -a option to the close_filesys command which writes any changes
to the superblock or block group descriptors to all of the backup
superblock locations.
Theodore Ts'o [Thu, 28 May 2009 21:05:40 +0000 (17:05 -0400)]
e2fsck: if i_file_acl_hi is non-zero, fix it and don't abort e2fsck -p
Ext3 filesystems don't care if i_file_acl_hi is non-zero in some
inode, and newer kernels should ignore this field (although 2.6.29 and
older kernels will not). So e2fsck should fix this without aborting
an e2fsck preen operation.
number9652 [Tue, 19 May 2009 20:34:12 +0000 (13:34 -0700)]
libext2fs: add ext2fs_extent_open2
The patch below adds a function, ext2fs_extent_open2(), that behaves
as ext2fs_extent_open(), but will use the user-supplied inode
structure when opening an extent instead of reading the inode from
disk. It also changes several of the calls to extent_open() to use
this enhancement.
Signed-off-by: Nic Case <number9652@yahoo.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Ken Chen [Thu, 28 May 2009 13:55:10 +0000 (09:55 -0400)]
Add empty function for init_resource_track() and print_resource_track()
in the case of ! defined RESOURCE_TRACK, so that we can clean up #ifdef
throughout e2fsck source.
Signed-off-by: Ken Chen <kenchen@google.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Ken Chen [Tue, 5 May 2009 03:12:30 +0000 (20:12 -0700)]
e2fsck: add resource tracking for passes 1[b-d]
On ext2, time tracking for pass1 includes both error detection and
specific type of fs fix-up phase (e.g. block referenced by multiple
inodes). The multi-reference fix-up phase some time take significant
amount of time to complete. We would like to track time spent in sub
component of pass1 by having a finer granularity during pass1b through
pass1d phase.
Signed-off-by: Ken Chen <kenchen@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Thu, 28 May 2009 04:23:43 +0000 (23:23 -0500)]
chattr: manpage tidyups
Tidy up the chattr(1) manpage to completely document all
available options, and differentiate those which are read-only
early in the manpage as well.
* Remove "I" from settable attribute list
* add "e" to 2nd list of settable attributes & descriptions
* Note that h/E/I/X/Z are readonly
* Correct "H" to "h" for huge file attribute description
* fix long_name for indexed directory in flags_array
Addresses-Red-Hat-Bugzilla: BZ#502971
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This patch adds new option, +e to chattr. The +e option
is used to convert the ext3 format (non extent) file
to ext4 (extent) format. This can be used to migrate
the ext3 file system to ext4 file system.
Andreas Dilger [Thu, 28 May 2009 06:47:37 +0000 (00:47 -0600)]
e2fsck: use unsigned values for memory tracking stats
Use unsigned values for printing memory tracking to avoid overflows.
The mallinfo() data is currently signed ints, but it might change in
the future so we may as well compute/print unsigned longs.
Signed-off-by: Andreas Dilger <adilger@sun.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Mon, 18 May 2009 22:02:32 +0000 (17:02 -0500)]
resize2fs: fix minimum size calculations
The extra padding added to the minimum size calculations:
/*
* We need to reserve a few extra blocks if extents are
* enabled, in case we need to grow the extent tree. The more
* we shrink the file system, the more space we need.
*/
if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS)
blks_needed += (fs->super->s_blocks_count - blks_needed)/500;
can go quite wrong if we've already added up more "blks_needed"
than our current size, and the above subtraction wraps. This can
easily happen for a filesystem which is almost completely full.
In this case, just return the current fs size as the minimum and
be done with it.
With this fix we could probably call calculate_minimum_resize_size()
for each resize2fs invocation and refuse to resize smaller than that?
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Wed, 20 May 2009 21:36:26 +0000 (16:36 -0500)]
resize2fs: fix ENOSPC corruption case
http://people.redhat.com/esandeen/livecd-creator-imagefile.bz2
contains an image (for now) which, when resized to 578639, corrupts
the filesystem.
This is a bit crazy, I guess, because the fs currently has only
1 free block, but still, we should be graceful about the failure.
Perhaps it would make sense to check the requested valuea against
the minimum value resize2fs would compute for "-P" and fail (at
least without a force).
But in any case, this exposed 2 bugs when moving that one block
required an extent split, which is what hit the ENOSPC.
For starters, ext2fs_extent_set_bmap() in the "(re/un)mapping last
block in extent" case was replacing the old extent before the
new one was created; when the new extent creation failed, it
left us in an inconsistent state. Simply changing the order of
the two should fix this problem.
Next, ext2fs_extent_insert was calling ext2fs_extent_delete()
on *any* error, including one caused by failure to allocate a new
block to split the node to hold that extent ... the handle was left
unchanged, and we deleted the -original- extent.
As a quick fix for this, just don't do the delete if we fail the split,
though this may need to be smarter. I don't think we have terribly
consistent behavior about where a handle is left on various errors.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 26 May 2009 00:14:04 +0000 (20:14 -0400)]
e2fsck: Fix journal replay bug which reverts changes to the bg descriptors
Fix a regression in e2fsprogs 1.41.5 which would undo updates to the
block group descriptors after a journal replay, caused by commit b7c5b403. We now use ext2fs_free() instead of ext2fs_close() to make
sure we the library will never try to write out superblock or block
group descriptors.
Andreas Dilger [Mon, 18 May 2009 03:03:04 +0000 (23:03 -0400)]
e2fsck: initialize error handling before journal replay
One of our customers hit a temporary IO error during an e2fsck run during
the read from the journal. It seems that the read error resulted in
e2fsck automatically discarding the journals and recreating them on several
filesystems on this node without any prompting from the user:
end_request: I/O error, dev sdg, sector 484832
Buffer I/O error on device sdg, logical block 60604
fsck-sdg[8276]: ls2-OST024c: Superblock has an invalid ext3 journal (inode 8).
fsck-sdg[8276]: CLEARED.
fsck-sdg[8276]: *** ext3 journal has been deleted - filesystem is now ext2
only ***
fsck-sdg[8276]: ls2-OST024c was not cleanly unmounted, check forced.
fsck-sdg[8276]: ls2-OST024c: Journal inode is not in use, but contains data.
CLEARED.
fsck-sdg[8276]: ls2-OST024c: Recreate journal to make the filesystem ext3
again?
fsck-sdg[8276]: FIXED.
fsck-sdg[8276]: Creating journal (32768 blocks): Done.
fsck-sdg[8276]:
fsck-sdg[8276]: *** journal has been re-created - filesystem is now ext3 again
***
fsck-sdg[8276]: ls2-OST024c: 39818/20183248 files (8.2% non-contiguous), 222122257/779902976 blocks
fsck-sdg[8276]: exit code 1 (file system errors corrected)
The following patch moves the e2fsck error handler initialization earlier
in the e2fsck startup code before the journal is processed, so that the
user will be prompted for an action. This is the first IO that is not
part of ext2fs_open() where fs->io is first initialized.
It doesn't seem possible to initialize the error handlers for the initial
filesystem open without changing the prototype for ext2fs_open2(). If we
are getting a new ext2fs_open3() prototype for 64-bit it might make sense
to add at least "read_error" as a parameter ("write_error" is not strictly
necessary for the open and could be set afterward).
Signed-off-by: Andreas Dilger <adilger@sun.com> Signed-off-by: Jim Garlick <garlick@llnl.gov> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sun, 17 May 2009 12:42:52 +0000 (08:42 -0400)]
e2fsck: Don't crash if an inode with a bad extent header is not cleared
If ext2fs_extent_open() fails due to a corrupt extent header, and the
user declines to clear the inode, check_blocks_extents() should bail
out; otherwise, it will cause a core dump due a null pointer
dereference.
Karel Zak [Mon, 27 Apr 2009 13:00:57 +0000 (15:00 +0200)]
blkid: use /dev/mapper/<name> rather than /dev/dm-<N>
The libblkid (since v1.41.1) returns private device-mapper names (e.g.
/dev/dm-0). It's because the probe_one() function scans /dev before
/dev/mapper.
brw-rw---- 1 root disk 253, 0 2009-04-27 13:41 /dev/dm-0
brw-rw---- 1 root disk 253, 0 2009-04-27 13:41 /dev/mapper/TestVolGroup-TestLogVolume
Old version:
# blkid -t LABEL="TEST-LABEL" -o device
/dev/dm-0
Andreas Dilger [Tue, 28 Apr 2009 18:59:07 +0000 (12:59 -0600)]
e2fsck: cleanup whitespace in problem.c and problem.h
Cleanup whitespace in the problem.h and problem.c files. Removes a
bunch of places where tabs follow spaces, whitespace on empty lines, etc.
I didn't reformat the indenting of the entire problem.h error codes,
but there is some room for doing this...
Signed-off-by: Andreas Dilger <adilger@sun.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Mon, 27 Apr 2009 22:59:24 +0000 (16:59 -0600)]
e2fsck: Add test code in problem.c to verify problem codes
We've hit a number of cases where the error codes in problem.h have
been assigned duplicate values compared to problems in our own e2fsck
patches, and this can lead to confusing and difficult to find bugs
in e2fsck (e.g. wrong problem messages, incorrect repair action, etc).
Attached is a test case for the problem.c file to ensure that the
problem table is sorted and does not contain any duplicate values.
Having the problem table sorted allows the correctness checking to be
very simple, and if it ever became important for performance we could
use binary searching of the problem table for the specific problem code.
Signed-off-by: Andreas Dilger <adilger@sun.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2fsck: Skip journal checks if the fs is mounted and doesn't need recovery
If we are checking a mounted filesystem (typically the root
filesystem, mounted read/only) and the NEEDS_RECOVERY flag is not set,
skip all of the checks associated with making sure the journal is
consistent. There is the very slight possibility we could lose if the
NEEDS_RECOVERY flag was somehow cleared even though there was data in
the journal, but this has practically never happend in practice, and
it reduces the number of reads required at boot-time, which is a big
deal when trying to reduce boot times with HDD's.
libext2fs: read the block group descriptors more efficiently
When opening a filesystem, make ext2fs_open2() much more efficient by
reading the normal block group descriptors all at once, instead of one
block at a time.
e2fsck: Reduce unnecessary I/O when checking backup superblock
E2fsck needs to check to see if the backup superblock differs from the
primary superblock. Previously it was doing so by calling
ext2fs_open(), which does a lot of unnecessary work, including reading
all of the backup block group descriptors. Avoid this by reading in
the backup superblock directly.
e2fsck: Don't test the resize_inode if the filesystem is clean
Move check_resize_inode() out of check_super_block(), since we only
need to test the resize_inode for correctness only if the filesystem
requires checking. This change avoids a lot of I/O operations which
slows down a 1 second boot.
Eric Sandeen [Thu, 23 Apr 2009 03:51:51 +0000 (22:51 -0500)]
blkid: remove whole-disk entries from cache when partitions are found
We can get into a situation in blkid where whole disks remain
in the cache, even though partitions are found. For labels
such as sun disklabels which may have the first partition
beginning at sector 0, this is even somewhat likely.
1) create a sun disklabel w/partitions
2) mkfs the first partition (at sector 0)
3) remove the partition table
4) run blkid - this finds the fs on the whole disk, places in cache
5) recreate the partition table
6) run blkid - this finds the partition, places in cache
And now we have both /dev/sda and /dev/sda1 in cache.
There are heuristics in probe_all to avoid putting the whole disk
in cache if it has partitions, but there is nothing to remove the
whole-disk entry in the above case. I think the below patch
suffices, although I haven't quite convinced myself that setting
the lens[which]=0; is the right logic for that bit of state...
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the device name doesn't start with a slash, ignore the /etc/mtab
entry, so that relative pathnames passed into functions such as
ext2fs_check_mount_point() or ext2fs_check_if_mounted() don't return
false positives.
resize2fs: Print a warning message if the ftruncate system call fails
Resize2fs will attempt to truncate an image file of a filesystem down
to size for the convenience of the system administrator. If the
truncate operation fails, print a warning message. This also avoids a
gcc warning message.
Fixed a potential bug where by partial returns from the write(2)
system call could lost characters to be sent to external progress bar
display program.
libss: ss_execute_line: reflect any error codes from system() to the caller
This is primarily to silence a gcc warning, but it's better to reflect
the error from system() up to the caller. In this case we don't
actually use it for anything, but that's OK.
libcom_err: Declare prototypes for et_list_lock/unlock in com_err.h
Define the prototypes for et_list_lock() and et_list_unlock() in
com_err.h. This promotes better error checking and avoids warnings
when compiling the library and programs that call these functions.
libe2p: Declare prototypes for the journal feature name functions in e2p.h
Define the prototypes for e2p_jrnl_feature2string() and
e2p_jrnl_string2feature() in e2p.h. This promotes better error
checking and avoids warnings when compiling the library and programs
that call these functions.