Ashish Sangwan [Sat, 11 May 2013 03:42:13 +0000 (09:12 +0530)]
mke2fs: calculate journal blocks just after fs initialize
We can calculate journal blocks as soon as blocksize is set.
It will help to figure out wrong journal blocks count earlier.
This will save some un-necessary initialization.
Without patch output =>
mke2fs /dev/sdc1 -J size=1048576
mke2fs 1.42.7 (21-Jan-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
61312 inodes, 244936 blocks
12246 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=251658240
8 block groups
32768 blocks per group, 32768 fragments per group
7664 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
The requested journal size is 268435456 blocks; it must be
between 1024 and 10240000 blocks. Aborting.
Eric Sandeen [Wed, 19 Jun 2013 01:37:05 +0000 (20:37 -0500)]
resize2fs: move bitmaps if shrinking would orphan them
It is possible to have a flex_bg filesystem with block groups
which have inode & block bitmaps at some point well past the
start of the group.
If an offline shrink puts the new size somewhere between
the start of the block group and the (old) location of
the bitmaps, they can be left beyond the end of the filesystem,
i.e. result in fs corruption.
Check each remaining block group for whether its bitmaps
are beyond the end of the new filesystem, and reallocate
them in a new location if needed.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 16 Jun 2013 22:58:40 +0000 (18:58 -0400)]
Use ext2fs_cpu_to_be32() instead of cpu_to_be32() in kernel-jbd.h
Avoid compatibility problems by using the byte swapping functions
defined by e2fsprogs, instead of the ones defined in the system header
files. We use them everywhere else, so we should use them in
kernel-jbd.h too.
Theodore Ts'o [Sun, 16 Jun 2013 18:34:59 +0000 (14:34 -0400)]
Try to use secure_getenv() in preference to __secure_getenv()
If secure_getenv() use it in preference to __secure_getenv().
Starting with (e)glibc version 2.17, secure_getenv() exists, while
__secure_getenv() only works with shared library links (where it is a
weak symbol), but not for static links with /lib/libc.a
Andreas Dilger [Sun, 16 Jun 2013 01:45:37 +0000 (21:45 -0400)]
mke2fs: don't set root dir UID/GID automatically
Don't change the root directory's UID/GID automatically just because
mke2fs was run as a non-root user. This can be confusing for users,
and is not flexible for non-root installation tools that need to
create a filesystem with different ownership from the current user.
Add the "-E root_owner[=uid:gid]" option to mke2fs so that the user
and group can be explicitly specified for the root directory. If
the "=uid:gid" argument is not specified, the current UID and GID
are extracted from the running process, as was done in the past.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 15 Jun 2013 22:44:09 +0000 (18:44 -0400)]
tests: clean up sed filtering of test output
The sed filters for test outputs that are used to remove build and
test specific information (such as version strings, dates, times,
UUIDs) were unconditionally deleting the first line of output. This
would normally contain the tool version string, but in some cases
contained other information that was being lost. This can lead to
difficulty debugging test failures.
The sed filtering has been changed to only remove the actual version
strings. As well, similar filter strings were duplicated throughout
many scripts, and "sed" and "tr" were often called multiple times in
a pipeline. These have been consolidated into a single filter.sed
file to avoid having to maintain these filters in multiple places.
In a few cases, accidentally deleted messages have been restored to
the expect output for the tests. In other cases, trivial whitespace
has been changed in the expect files.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
David Jeffery [Fri, 7 Jun 2013 00:04:33 +0000 (20:04 -0400)]
e2fsck: detect invalid extents at the end of an extent-block
e2fsck does not detect extents which are outside their location in the
extent tree. This can result in a bad extent at the end of an extent-block
not being detected.
e2fsck does not detect this bad extent which both overlaps another, valid
extent, and is invalid by being beyond the end of the extent above it in
the tree.
This patch modifies e2fsck to detect this invalid extent and remove it.
Signed-off-by: David Jeffery <djeffery@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Eric Sandeen <sandeen@redhat.com>
E2fsck previously was complaining with zero-length extended attributes
if they appeared in the in-inode xattr space. Test to make sure
e2fsck is now happy with such xattrs.
libext2fs: only use override function when reading an 128 byte inode
The ext2fs_read_inode_full() function should not use fs->read_inode()
if the caller has requested more than the base 128 byte inode
structure and the inode size is greater than 128 bytes. Otherwise the
caller won't get all of the bytes that they were asking for, since
there's no way for the fs->read_inode override function can know what
the size of the buffer passed to ext2fs_read_inode_full().
tune2fs: disallow tune2fs to set inode size larger than block size
Disallow tune2fs command to set the inode size to be larger than the
block size. Without this patch, tune2fs makes the file system to be
unmountable.
Steps to reproduce:
1.Create ext4 without flex_bg (or just create ext3)
# mke2fs -t ext4 -O ^flex_bg DEV
2.Set inode size larger than block size
# tune2fs -I 8192 DEV
3. We failed to mount FS
# mount DEV MP
mount: wrong fs type, bad option, bad superblock on /dev/sda7,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
tests: create crcsum progam to support resizing tests
The only checksum program which we can reliably count upon being
installed on all systems is "sum", which is not a particular robust
checksum. The problem with using md5sum or sha1sum is it hat it may
not be installed on all systems. So create a crcsum program which is
used so we can validate that a data file on a resized file system has
not been corrupted.
Theodore Ts'o [Sun, 31 Mar 2013 17:24:39 +0000 (13:24 -0400)]
resize2fs: fix off-line resize of file systems with flex_bg && !resize_inode
When doing an off-line resize2fs of an initially very small file
system, it's possible to run out of reserved gdt blocks (which are
reserved via the resize inode). Once we run out, we need to move the
allocation bitmaps and inode table out of the way to grow the gdt
blocks. Unfortunately, when moving these metadata blocks, it was
possible that a block that had been just been newly allocated for a
new block group could also get allocated for a metadata block for an
existing block group that was being moved.
To prevent this, after we grow the gdt blocks and allocate the
metadata blocks for the new block groups, make sure all of these
blocks are marked as reserved.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reported-by: John Jolly <john.jolly@gmail.com>
Theodore Ts'o [Wed, 13 Mar 2013 18:19:16 +0000 (14:19 -0400)]
debugfs: fix command failures for extent_inode commands that take arguments
The extent_inode commands split_node, replace_node, and insert_node
take arguments which resulted in confusing error messages after
succeeding. Fix this.
Jan Kara [Mon, 25 Feb 2013 05:55:05 +0000 (05:55 +0000)]
e2p: Fix 's' handling in parse_num_blocks2()
parse_num_blocks2() wrongly did:
num << 1;
when log_block_size < 0. That is obviously wrong as such statement has
no effect (and the compiler properly warns about it). Callers expect
returned value to be in bytes when log_block_size < 0 so fix the
statement accordingly.
Signed-off-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 23 Feb 2013 03:23:05 +0000 (22:23 -0500)]
chattr: allow clearing the extent flag
In order to support kernels which support conversion of extent-mapped
files to direct/indirect mapped files, remove the sanity check which
prevented clearing the extent flag in chattr. Kernels which don't
support this will simply give an Operation Not Supported error.
Theodore Ts'o [Mon, 28 Jan 2013 14:02:23 +0000 (09:02 -0500)]
debugfs: add sanity check to make sure we never shift 64 bits right
In the tables which are used to parse the fields for the set_fields
command, there should never be a entry which has a size set to 8
bytes, and two pointers defined. Not only would it result in
undefined behavior in the compiled code, it doesn't make any sense and
is definitely a bug.
Reported-by: Xi Wang <xi.wang@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Phillip Susi [Thu, 24 Jan 2013 16:21:56 +0000 (11:21 -0500)]
libext2fs: fix ext2fs_llseek on i386
ext2fs_llseek() was using lseek instead of lseek64. The
only time it would use lseek64 is if passed an offset that
overflowed 32 bits. This works for SEEK_SET, but not
SEEK_CUR, which can apply a small offset to move the file
pointer past the 32 bit limit.
The code has been changed to instead try lseek64 first, and
fall back to lseek if that fails. It also was doing a
runtime check of the size of off_t. This has been moved to
compile time.
This fixes a problem which would cause e2image when built for
x86-32 to bomb out when used with large file systems.
Theodore Ts'o [Mon, 21 Jan 2013 22:15:25 +0000 (17:15 -0500)]
contrib: add thread locking and readdir64_r support to spd_readdir
This is part of a series of improvements from a 2008 version of
spd_readdir.c that somehow didn't make it into the version which we
checked into e2fsprogs git tree.
Theodore Ts'o [Mon, 21 Jan 2013 21:35:16 +0000 (16:35 -0500)]
contrib: add safe_getenv() support to spd_readdir
This is part of a series of improvements from a 2008 version of
spd_readdir.c that somehow didn't make it into the version which we
checked into e2fsprogs git tree.
Theodore Ts'o [Sun, 20 Jan 2013 04:21:11 +0000 (23:21 -0500)]
libext2fs: teach the ext2fs_*_block_bitmap_range2() about clusters
The ext2fs_{mark,unmark,test}_block_bitmap2() functions understand
about clusters, and will take block numbers and convert them to
clusters before checking the bitmap. The
ext2fs_*_block_bitmap_range2() functions did not do this, which made
them inconsistent. Fortunately, nothing has depended on this
incorrect behavior, and in fact most of the usage of these functions
have only recently been added, and only for optimizations that were
only enabled for non-bigalloc file systems.
So this is a change in previously exported functions, but (a) it
doesn't change the behavior at all for non-bigalloc file systems, and
(b) the change is more likely to fix bugs for bigalloc file systems.
For example, this change fixes a problem with resize2fs and bigalloc
file systems.
Theodore Ts'o [Wed, 16 Jan 2013 18:43:36 +0000 (13:43 -0500)]
resize2fs: check in test-resize script
This should be made into a more formal, automated test case, but for
now, save this as script since it's useful for validating resize2fs's
handling of very large file systems.
Phillip Susi [Tue, 15 Jan 2013 20:31:23 +0000 (15:31 -0500)]
e2image: add -a switch to include all data
Normally the raw and QCOW2 images only contain fs metadata.
Add a new switch ( -a ) to include all data. This makes it
possible to use e2image to clone a whole filesystem.
Darren Hart [Fri, 4 Jan 2013 20:00:59 +0000 (12:00 -0800)]
debugfs: add symlink command
Add support for symbolic links using a new symlink command. Modeled
after the do_mkdir() command.
Testing demonstrates both fastlinks and slowlinks work correctly.
Very long target paths fail as the command parsing appears to truncate
the input to somewhere around 256 bytes.
Signed-off-by: Darren Hart <dvhart@infradead.org> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Cc: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: Andreas Dilger <adilger@dilger.ca>
Darren Hart [Fri, 4 Jan 2013 20:00:58 +0000 (12:00 -0800)]
libext2fs: add the ext2fs_symlink() function
Creating symlinks is a complex affair when accounting for slowlinks.
Create a new function, ext2fs_symlink(), modeled after ext2fs_mkdir().
Like ext2fs_mkdir(), ext2fs_symlink() takes on the task of allocating a
new inode and block (for slowlinks), setting up sane default values in
the inode, copying the target path to either the inode (for fastlinks)
or to the first block (for slowlinks), and accounting for the inode and
block stats. Disallow link targets longer than blocksize as the Linux
kernel prevents this.
It does not attempt to expand the parent directory, instead returning
EXT2_ET_DIR_NO_SPACE and leaving it to the caller to expand just as
ext2fs_mkdir() does. Ideally, I think both of these functions should
make a single attempt to expand the directory.
[ Fixed a few bugs discovered when creating a test case for ext2fs_symlink() ]
Signed-off-by: Darren Hart <dvhart@infradead.org> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Cc: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: Andreas Dilger <adilger@dilger.ca>
Theodore Ts'o [Tue, 15 Jan 2013 00:29:54 +0000 (19:29 -0500)]
libext2fs: avoid 32-bit overflow in ext2fs_initialize with a 512M cluster size
If the user attemps to create a 512MB cluster, we need to adjust the
defaults to avoid a 32-bit overflow of s_blocks_per_group. Also check
to make sure that the caller of ext2fs_initialize() has not given a
value of s_clusters_per_group that would result in an overflow of
s_blocks_per_group.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Theodore Ts'o [Tue, 15 Jan 2013 00:03:11 +0000 (19:03 -0500)]
mke2fs: teach mke2fs to understand -b 4k and -C 256M
The -b and -C options now use parse_num_blocks2() instead of strtol,
so that users can specify -C 256M instead of the much less convenient
-C 268435456.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Theodore Ts'o [Mon, 14 Jan 2013 22:30:23 +0000 (17:30 -0500)]
mke2fs: the -g option will now specify the clusters per block group
If bigalloc is enabled, then -g will specify the clusters per block
group. (If bigalloc is not enabled, then a cluster == a block, so the
meaning of -g is not changed.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Zheng Liu [Sun, 13 Jan 2013 09:08:13 +0000 (17:08 +0800)]
mke2fs: require the bigalloc feature explicity if the cluster size is set
When cluster-size is specified without the bigalloc feature, mke2fs
will ignore this argument silently. But user might think bigalloc
feature has been enabled unless they use the dumpe2fs command. So now
we ask user to set bigalloc feature explicity when cluster-size is
enabled. This can make sure that users understand what they are doing
because bigalloc might impact the performance for some workloads.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sat, 12 Jan 2013 22:33:44 +0000 (17:33 -0500)]
resize2fs: add debugging code to test the old online resizing interfaces
The old online resize ioctl interfaces are still present in the
kernel, and we want to make it easy to test both the kernel code for
those older interfaces, and resize2fs's use of those interfaces in a
relatively easy manner.
To do this, resize2fs will now check the environment variable
RESIZE2FS_KERNEL_VERSION. If the version given is less than 3.3, then
do not try using the new resizing ioctl, but instead use the resizing
ioctls that were used before Linux version 3.3.
If the version given is less than 3.7, then emulate sanity checks
which get done to protect against the fact that the new resizing ioctl
prior to 3.7 did not handle meta_bg resizing. (This was previously
tested via the presence of the RESIZE2FS_NO_META_BG_RESIZE environment
variable. But the new environment variable, RESIZE2FS_KERNEL_VERISON,
is more general.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
Eric Whitney [Fri, 4 Jan 2013 10:27:12 +0000 (10:27 +0000)]
config: silence printf format warnings
The printfs in the asm_types.c code contained within parse-types.sh
expect sizeof to return an int. Fix this for architectures where this
isn't true (x86_64, etc.) so we don't see warning messages while
running the configure script.
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Wed, 9 Jan 2013 01:47:11 +0000 (20:47 -0500)]
libext2fs: ext2fs_open2() should not set ret_fs after a MMP failure
The addition of MMP code was added in the wrong place, so ret_fs could
get set (and EXT2_FLAG_NOFREE_ON_ERROR was cleared as well, which
could confuse e2fsck which depends on this flag being cleared if
ext2fs_open2() succeeded.)
e2fsck: avoid memory corruption on ext2fs_open2 failure
In try_open_fs(), if ext2fs_open2() returns an error, do not try to
access the struct ext2_filesys. The previous check 'if (ret_fs)' was
always true, but even 'if (*ret_fs)' might be incorrect in some cases,
so check 'retval==0' instead.
Previously e2fsck would corrupt memory if the log file name was longer
than 100 bytes (e.g., a long log_filename value in e2fsck.conf or a
pattern that expands out to more than 100 bytes). This was due to
incorrectly calling realloc() in append_string() on the struct string
instead of the malloc'ed char* buffer, among other problems. This
patch fixes the call to realloc() and also ensures that the buffer is
grown by sufficiently many bytes (not just by 2x).
Zheng Liu [Sun, 6 Jan 2013 12:25:17 +0000 (20:25 +0800)]
mke2fs: check extents feature when bigalloc feature is enabled
When bigalloc feature is enabled in mkfs, extents feature also needs
to be enabled. But now when bigalloc feature is enabled without
extents feature, users will not get any warning messages until they
try to mount this file system.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There are a number of places where we multiply a dgrp_t with
s_blocks_per_group expecting that we will get a blk64_t. This
requires a cast, or using the convenience function
ext2fs_group_first_block2().
This audit was suggested by Eric Sandeen.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Theodore Ts'o [Thu, 3 Jan 2013 13:35:25 +0000 (08:35 -0500)]
resize2fs: use [un]mark_block_range bitmap functions to reduce CPU usage
Use ext2fs_[un]mark_block_range2() functions to reduce the CPU
overhead of resizing large file systems by 45%, primarily by
reducing the time spent in fix_uninit_block_bitmaps().
Theodore Ts'o [Thu, 3 Jan 2013 14:01:22 +0000 (09:01 -0500)]
resize2fs: fix 32-bit overflow when calculating the number of free blocks
This caused the free blocks count in the superblock to be incorrect
after resizing a 64-bit file system if the number of free blocks
overflowed a 32-bit value.
Theodore Ts'o [Thu, 3 Jan 2013 04:31:04 +0000 (23:31 -0500)]
resize2fs: fix 32-bit overflow issue which can corrupt 64-bit file systems
Fix a 32-bit overflow bug caused by a missing blk64_t cast which can
cause the block bitmap to get corrupted when doing an off-line resize
of a 64-bit file system.
Theodore Ts'o [Wed, 2 Jan 2013 15:06:09 +0000 (10:06 -0500)]
Clean up texinfo files
Fix up the com_err.texinfo file so it will produce a valid printed
output, by cleaning up some errors in the texinfo file, and updating
texinfo.tex to be consistent with the version in the doc subdirectory.
Also add rules so we can generate pdf and ps files from
com_err.texinfo and libext2fs.texinfo.
Andreas Dilger [Thu, 22 Nov 2012 15:06:12 +0000 (15:06 +0000)]
filefrag: improvements to filefrag FIEMAP handling
Update the filefrag program to allow displaying the extents in
some different formats. Try and stay within 80 columns.
* add -k option to print extents in kB-sized units (like df -k)
* add -b {blocksize} to print extents in blocksize units
* add -e option to print extent format, even when FIBMAP is used
* add -X option to print extents in hexadecimal format
Internally, the FIBMAP handling code has been moved into its own
function like FIEMAP, so that the code is more modular. Extent
offsets are now handled in bytes instead of in blocks, to allow
printing extents with arbitrary block sizes. The extent header
printing also moved into its own function so that it can be shared
between the FIEMAP and FIBMAP handling routines, since it got more
complex with the different output options.
Only print error about FIBMAP being root-only a single time.
Print the filesystem type if it changes between specified files.
Add fsync() for FIBMAP if "-s" is given.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Thu, 29 Nov 2012 12:47:54 +0000 (05:47 -0700)]
libquota: quiet log_err() bad format warnings
The macro for log_err() was written so that it needed to always
have an argument, but GCC was unhappy to have an argument when
none was specified in the format string. Use the CPP "##" to
eat the preceeding comma if no argument is specified.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
projects / thirdparty / e2fsprogs.git / log
