Andreas Dilger [Thu, 16 Jun 2011 02:17:38 +0000 (22:17 -0400)]
check: build all dependencies before "make check"
If "make check" is run on a clean repository, it fails due to missing
dependencies for building the test programs. Have "make check" build
all dependencies before starting the tests to ensure that it can finish
without error.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 16 Jun 2011 02:11:59 +0000 (22:11 -0400)]
libext2fs: fix regression in ext2fs_new_block2() for uninit_bg file systems
Commit b0ecb787ef introduced a bug in check_block_uninit(), which is
used by ext2fs_new_block2(). This bug resulted in the block bitmap
for the block group in question not having space reserved for the file
system metadata blocks.
Andreas Dilger [Sat, 11 Jun 2011 16:51:44 +0000 (12:51 -0400)]
e2fsprogs: ignore generated files for "git status"
Ignore files generated by "make" or "make check" in "git status".
Ignore backup files from editing files and generated tags files.
Delete a temporary file in tests/d_loaddump/script at test cleanup.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 16:51:21 +0000 (12:51 -0400)]
ext2fs: fix error handling in ext2fs_add_dir_block
In ext2fs_add_dir_block() the dblist allocation size was changed to
grow as the number of items in the dblist increases. However, the
error handling in case of allocation failure wasn't changed to match.
Fix the error case to revert to the old allocation size on failure.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 16:22:36 +0000 (12:22 -0400)]
mke2fs: add extent-mapped journal regression test
All of the regression tests in e2fsprogs still use a block-mapped
journal (if any journal at all). Add a simple regression test that
tests extent-mapped journals for both mke2fs and e2fsck.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 16:17:29 +0000 (12:17 -0400)]
mke2fs: add test for large journal with lazy init
Add test for internal journal over 4GB in size, using the
lazy_journal_init and lazy_itable_init features. Otherwise
the filesystem metadata would be too large to reliably run on
test systems, and take too long to create/check the filesystem.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 16:17:29 +0000 (12:17 -0400)]
mke2fs: skip zeroing journal blocks
Add the ability to skip zeroing journal blocks on disk. This can
significantly speed up mke2fs with large journals. At worst the
uninitialized journal is only a very short-term risk (if at all),
because the journal will be overwritten on any new filesystem as
soon as any significant amount of data is written to disk, and
the new journal TID would need to match the offset/TID of an old
commit block still left on disk.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 16:17:29 +0000 (12:17 -0400)]
ext2fs: Handle internal journal over 2^32 bytes
The write_journal_inode() code is only setting the low 32-bit i_size
for the journal size, even though it is possible to specify a journal
up to 10M blocks in size. Trying to create a journal larger than 2GB
will succeed, but an immediate e2fsck would fail. Store i_size_high
for the journal inode when creating it, and load it upon access.
Use s_jnl_blocks[15] to store the journal i_size_high backup. This
field is currently unused, as EXT2_N_BLOCKS is 15, so it is using
s_jnl_blocks[0..14], and i_size is in s_jnl_blocks[16].
Rename the "size" argument "num_blocks" for the journal creation functions
to clarify this parameter is in units of filesystem blocks and not bytes.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 15:50:01 +0000 (11:50 -0400)]
ext2fs: don't use O_DIRECT if not available
O_DIRECT is not defined on OSX. Since direct IO is only a new
optimization and not needed for correct functionality, disable
it if O_DIRECT is unavailable.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 15:50:01 +0000 (11:50 -0400)]
misc: fix compile warnings on OSX
The BLKFLSBUF and FDFLUSH ioctls are Linux specific, and do not
really have anything to do with __GNUC__ (which is also used on
OS/X and Solaris). Only print these warnings on Linux systems.
statfs64() is deprecated on OSX and generates a deliberate warning.
Fix some other warnings that show up on OSX builds.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 15:33:02 +0000 (11:33 -0400)]
ext2fs: fix ext2fs_warn_bitmap32() return warning
This was reported as "control reaches end of non-void function",
but comparing to other similar functions it should be a void
function. Since it is only declared in the "private" ext2fsP.h
header, it should be OK to change the function prototype.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 15:29:54 +0000 (11:29 -0400)]
debugfs: fix icheck finding blocks used for xattrs
This was an "uninitialized variable" warning, but it turns out to be
a real bug. Without this change, it is not possible to use "icheck"
to find blocks that are used for the i_file_acl (xattr) block.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 11 Jun 2011 15:04:42 +0000 (11:04 -0400)]
ext2fs: fix undeclared posix_memalign() warning
Older distros do not define posix_memalign() by default in the
headers. If ext2fs.h is included early in the headers, it is
possible to "#define _XOPEN_SOURCE 600" so that the stdlib.h
header will define it, but if ext2fs.h is included after stdlib.h
there is no posix_memalign() declaration.
Add a posix_memalign() declaration if stdlib.h didn't do it. This
is a bit of a hack for GNU headers, but it works on Linux and OS/X
without problems.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 10 Jun 2011 22:51:58 +0000 (18:51 -0400)]
libext2fs: teach block allocation functions about bigalloc/clusters
Optimize ext2fs_new_block2() and ext2fs_get_free_blocks2() when
bigalloc is enabled. Also fix the uninitialized block bitmap code so
that it correctly deals clustered allocation.
Theodore Ts'o [Fri, 10 Jun 2011 21:55:09 +0000 (17:55 -0400)]
libext2fs: teach bitmap functions about bigalloc/cluster
This patch makes the following changes:
* ext2fs_allocate_block_bitmap() now allocates a bitmap with cluster
granularity for bigalloc file systems. For mke2fs and e2fsck, a
newly added function, ext2fs_allocate_subcluster_bitmap() allocates
a bitmap with block granularity (even for bigalloc file systems).
The newly added function ext2fs_get_bitmap_granularity() will return
the number of bits (log2) of the granularity used by the bitmap.
* The ext2fs_{mark,unmark,test}_block_bitmap2() functions will shift
their passed-in argument by log2(cluster_ganularity) bits right.
This means that the arguments for the single-argument bitmap
functions will be interpreted with block granluarity, since this
minimizes code changes in the rest of the code base.
* The ext2fs_{get,set}_block_bitmap_range() functions will interpret
their arguments in cluster granularity. This is a bit inconsistent,
but the caller of those functions will need to be taught about the
subtleties of clusters for bigalloc file systems.
Andreas Dilger [Tue, 7 Jun 2011 16:22:29 +0000 (10:22 -0600)]
mke2fs: Don't erase flash device if "-n" is given
If "mke2fs -n" is used, there should be no changes to the underlying
device. Unfortunately, when the "discard" option was added in commit c7cd908be59f48c66b4f3ac9a631ffe3dde4f1ab, it did not check for the "-n"
flag, and will discard all data on a flash device even if "-n" is given.
Check for the "noaction" flag before discarding any filesystem data.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 4 Jun 2011 14:20:47 +0000 (10:20 -0400)]
libext2fs: change fs->clustersize to fs->cluster_ratio_bits
The log2 of the ratio of cluster size to block size is far more useful
than just storing the cluster size. So make this change, and then
define basic utility macros: EXT2FS_CLUSTER_RATIO(),
EXT2FS_CLUSTER_MASK(), EXT2FS_B2C(), EXT2FS_C2B(), and
EXT2FS_NUM_B2C().
Theodore Ts'o [Sat, 4 Jun 2011 20:40:26 +0000 (16:40 -0400)]
libext2fs: change EXT2_MAX_BLOCKS_PER_GROUP() to be cluster size aware
Change the EXT2_MAX_BLOCKS_PER_GROUP so that it takes the cluster size
into account. This way we can open bigalloc file systems without
ext2fs_open() thinking that they are corrupt.
Theodore Ts'o [Sat, 4 Jun 2011 20:36:19 +0000 (16:36 -0400)]
libext2fs: require cluster size == block_size when opening a !bigalloc fs
In ext2fs_open() check to make sure the cluster size superblock field
is the same as the block size field when the bigalloc feature is not
set. This is necessary since we will start introducing calculations
based on the cluster size field.
Eric Sandeen [Tue, 31 May 2011 23:59:56 +0000 (19:59 -0400)]
e2fsck: don't check/clone duplicate xattr blocks in fs without xattr feature
I had an extremely corrupted customer filesystem which, after thousands
of lines of e2fsck output, found one more problem on an immediately
subsequent e2fsck. In short, a file had had its i_file_acl block
cloned due to being a duplicate. That ultimately got cleared
because the fs did not have the xattr feature, and the inode
was subsequently removed due to invalid mode.
The 2nd e2fsck pass found the cloned xattr block as in use, but
not owned by any file, and had to fix up the block bitmaps.
Simply skipping the processing of duplicate xattr blocks on a
non-xattr filesystem seems reasonable, since they will be cleared
later in any case.
(also fix existing brace misalignment)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2fsprogs: Unify the upper limit of reserved blocks count
In e2fsprogs, the upper limit of reserved blocks count is a half of
filesystem's blocks count. This patch fixes the incorrect checks of
reserved blocks count.
Eric Sandeen [Mon, 4 Apr 2011 19:11:52 +0000 (15:11 -0400)]
mke2fs: don't set stripe/stride to 1 block
Block devices may set minimum or optimal IO hints equal to
blocksize; in this case there is really nothing for ext4
to do with this information (i.e. search for a block-aligned
allocation?) so don't set fs geometry with single-block
values.
Zeev also reported that with a block-sized stripe, the
ext4 allocator spends time spinning in ext4_mb_scan_aligned(),
oddly enough.
Lukas Czerner [Wed, 18 May 2011 11:36:55 +0000 (13:36 +0200)]
tests: New i_e2image test to validate image creation/conversion
This commit adds new regression test called i_e2image which should
validate expected behaviour of e2image raw and qcow2 image creation
and qcow2 -> raw image conversion. You can run it with "make check" as
the rest of regression tests.
Testing is performed on three filesystem images with different block
sizes (1024, 2048, 4096). It creates raw and qcow2 images from the
original and then convert qcow2 image back to raw image. The results are
checksummed and compared with pre-prepared results. All md5sums should
stay the same and raw image created from original and qcow2 image should
be the same as well (just for the record).
Lukas Czerner [Wed, 18 May 2011 12:20:47 +0000 (14:20 +0200)]
e2image: Support for conversion QCOW2 image into raw
This commit adds support for converting QCOW2 image created previously
with e2image into raw image. The QCOW2 image is detected automatically,
so there is not new option. Just use following command:
e2image -r image.qcow image.raw
No that this tool is aimed to quickly convert qcow2 image created with
e2image into raw image. In order to improve speed we are doing some
assumption I believe might not be true for regular qcow2 images. So it
was not tested with regular QCOW2 images and it might not work with
them. The intention of this tool is only convert images previously
created by e2image.
Note that there is nothing special with QCOW2 images created by e2images
and it can be used with tools like qemu-img, or qemu-nbd without any
problems.
Lukas Czerner [Wed, 18 May 2011 11:36:53 +0000 (13:36 +0200)]
e2image: Add support for qcow2 format
This commit adds support for exporting filesystem into QCOW2 image
format. Like sparse format this saves space, by writing only necessary
(metadata blocks) into image. Unlike sparse image, QCOW2 image is NOT
sparse, hence does not change its size by copying with not-sparse-aware
tools.
New options '-Q' has been added to tell the e2image to use QCOW2 as an
output image format. QCOW2 supports encryption and compression, however
e2image so far does no support such features, however you can still
scramble filenames with '-s' option.
Lukas Czerner [Wed, 18 May 2011 12:19:52 +0000 (14:19 +0200)]
e2fsprogs: Add memory allocation and zero-out helpers
Add functions ext2fs_get_memzero() which will malloc() the memory
using ext2fs_get_mem(), but it will zero the allocated memory afterwards
with memset().
Add function ext2fs_get_arrayzero() which will use calloc() for
allocating and zero-out the array.
Aditya Kali [Tue, 10 May 2011 21:51:31 +0000 (14:51 -0700)]
mke2fs: Allow specifying reserved_ratio via mke2fs.conf
This patch adds support for specifying 'reserved_ratio' (percent blocks
reserved for super user, same as '-m' command line option) in mke2fs.conf.
It adds profile_get_double function in profile.c that allows reading
floating point values from profile files.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
It will cause segfault, but it is caused by another issue. See my
patch "mke2fs: Avoid segmentation fault in
ext2fs_alloc_generic_bmap". And with that patch, the mkfs.ext4 will
fail with the error: /dev/sdb3: Memory allocation failed while setting
up superblock
The reason is that in ext2fs_initialize, we align s_inodes_per_group
to 8, but fails to consider the case that s_inodes_per_group < 8.
So make at least 8 inodes for s_inodes_per_group.
Signed-off-by: Tao Ma <boyu.mt@taobao.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lukas Czerner [Mon, 21 Mar 2011 10:14:00 +0000 (11:14 +0100)]
e2fsprogs: fix memory leak in ext2fs_free_generic_bmap()
In ext2fs_free_generic_bmap() when we are freeing 64-bit bitmap, we do
call free_bmap() to free backend specific bitmap structures, however we
should also free ext2fs_generic_bitmap structure as well.
Lukas Czerner [Mon, 21 Mar 2011 10:13:41 +0000 (11:13 +0100)]
e2fsprogs: fix freeing bitmap in allocation error path
In ext2fs_alloc_generic_bmap() error path, when new bitmap allocation
fails ext2fs_generic_bitmap should be freed, however in current state it
first frees ext2fs_generic_bitmap and then
ext2fs_generic_bitmap->description which is wrong. This commit fix the
free ordering.
Theodore Ts'o [Sun, 8 May 2011 03:14:06 +0000 (23:14 -0400)]
e2fsck: make the "fs is mounted; continue?" prompt more paranoid
A user received the "file system is mounted; do you really want to
continue" prompt, and then instead of typing "n" for no, forgot that
he hadn't declined to continuation question, and typed the up-arrow
key, which in his locale, the 'A' in "^[[A" was interpreted as "yes",
and he lost data.
This was clearly the user's fault, but to make e2fsck a bit safer
against user stupidity/carelessness, we will change the "fs is
mounted; continue?" prompt to default to no, and treat the escape
character (along with the return and space characters, currently) as a
request for the default answer.
Eric Sandeen [Thu, 5 May 2011 18:21:08 +0000 (13:21 -0500)]
filefrag: count 0 extents properly when verbose
/boot/a: 0 extents found
works properly, but
Filesystem type is: ef53
Filesystem cylinder groups is approximately 61
File size of a is 0 (0 blocks, blocksize 1024)
ext logical physical expected length flags
a: 1 extent found
yields 1 extent when it should be 0.
Fix this up by special-casing no extents returned in verbose
mode; skip printing the header for the columns too, since there
are no columns to print.
Also, in nonverbose mode we can set fm_extent_count to 0
so that FIEMAP will just query the extent count without gathering
details; clarify this with a comment.
Addresses-RedHat-Bugzilla: 653234 Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 18 Mar 2011 18:47:15 +0000 (14:47 -0400)]
add new superblock field: s_overhead_blocks
It turns out that it's very hard to calculate overheads in the face of
clustered allocation (bigalloc). This is because multiple metadata
blocks from different block groups can end up in the same allocation
cluster. Calculating the exact overhead requires O(all block bitmaps)
in memory, or O(number of block groups**2) in time. So we will
calculate this at mkfs time and stash it in the superblock.
Theodore Ts'o [Mon, 28 Feb 2011 01:09:54 +0000 (20:09 -0500)]
mke2fs: If the device supports discard, don't print an error message
Check to see if the device supports discard before starting the
progress bar, and then printing an error about inappropriate ioctl for
device (when creating a file system image to a file, for example).
Also, add a function signature in the ext2_io.h header file for
io_channel_discard() and fix an extra, uneeded argument in mke2fs's
call to that function.
Theodore Ts'o [Sat, 26 Feb 2011 02:43:54 +0000 (21:43 -0500)]
Add basic BIGALLOC support for cluster-based allocation
This adds the superblock fields needed so that dumpe2fs works and the
code points and renames the superblock fields from describing
fragments to clusters.
Aditya Kali [Tue, 15 Feb 2011 22:27:27 +0000 (14:27 -0800)]
e2fsprogs: reserving code points for new ext4 quota feature
This patch adds support for detecting the new 'quota' feature in ext4.
The patch reserves code points for usr and group quota inodes and also
for the feature flag EXT4_FEATURE_RO_COMPAT_QUOTA.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lukas Czerner [Mon, 24 Jan 2011 19:52:00 +0000 (20:52 +0100)]
mke2fs: Display progress report during the device discard
For some time now we are doing initial discard of the device prior to
filesystem creation. However, there is no feedback for the user and
hence on some devices with slow TRIM implementation it may appear that
mke2fs is stuck.
This commit introduce new function mke2fs_discard_device(), which is a
wrapper for io_channel_discard(). The discard is done in chunks of
2GB, which seems reasonably well for both slow and fast devices, and
discard progress is reported back to the user.
I gave up on doing fancy things like align discard according to
discard_alignment, checking for discard granularity and computing
estimate time. First of all, because it would require either new ioctl
to retrieve those information or use of libudev library, none of it
seems to be worth it. Regarding discard_granularity, I doubt there is
any sane device with discard granularity that big it would affect this.
Eric Sandeen [Thu, 17 Feb 2011 21:56:17 +0000 (15:56 -0600)]
e2fsprogs: enable user namespace xattrs by default
User namespace xattrs are generally useful, and I think extN
is the only filesystem requiring a special mount option to
enable them, when xattrs are otherwise available. So this
change sets that mount option into the defaults, via a
mke2fs.conf option.
Note that if xattrs are config'd off, this will lead to a
mostly-harmless:
EXT4-fs (sdc1): (no)user_xattr options not supported
message at mount time...
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Thu, 17 Feb 2011 21:55:15 +0000 (15:55 -0600)]
e2fsprogs: turn off enforced fsck intervals by default
The forced fsck often comes at unexpected and inopportune moments,
and even enterprise customers are often caught by surprise when
this happens. Because a filesystem with an error condition will
be marked as requiring fsck anyway, I submit that the time-based
and mount-based checks are not particularly useful, and that
administrators can schedule fscks on their own time, or tune2fs
the enforced intervals if they so choose. This patch disables the
intervals by default, and I've added a new mkfs.conf option to
turn on the old behavior of random, unexpected, time-consuming
fscks at boot time. ;)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Wed, 16 Feb 2011 18:01:39 +0000 (12:01 -0600)]
e2fsprogs: create com_err.h link in includedir
After debian bug #192277, debian/rules started making a symlink
to com_err.h in /usr/include. Now I have Fedora bug #550889
for the same issue, and perhaps it's time to make this link
by default, rather than fixing it up in packaging steps?
[ Changed by tytso to remove the explicit -s option; this will default
to creating a hard link by default, which slightly faster. If
people want to use symlinks for all links during the install
process, they can use configure option --enable-symlink-install.
The reason for this change is that some file systems, like AFS,
don't support symlinks, and AFS users complain when they can't build
or install into AFS. So I don't want to use symlinks
unconditionally without a way of switching things back and forth,
and it's easier if we just make all links made during the install
process to be hard links or sym links. ]
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 20 Feb 2011 20:19:47 +0000 (15:19 -0500)]
badblocks: Fix up recover_block handling in badblocks
If there was a bad block for block #0, badblocks would never switch
back testing blocks more efficiently. In addition, we were
double-incrementing the blocks to be tested in the read/write test due
to failure to remove code.
Thanks to Ragnar Kjørstad for pointing these problems out.
Theodore Ts'o [Fri, 18 Feb 2011 03:58:21 +0000 (22:58 -0500)]
badblocks: Only report errors when reading/writing one block at a time
With Direct I/O, the kernel can report 0 bytes read even though the
first block has no errors. So there are any errors, we need try to
read/write blocks one at a time and to get an accurate report.
Kazuya Mio [Sun, 19 Dec 2010 22:01:24 +0000 (22:01 +0000)]
e4defrag: Use libext2fs to get the correct superblock information
Currently, e4defrag always does byte-swapping when it gets superblock
information, so the calculation of the best extents count is not
correct on little endian machine. This doesn't cause data corruption,
but it may confuse users by showing the wrong extent count. To solve
this problem, we use ext2fs_open() instead of get_superblock_info()
that is the original function.
Theodore Ts'o [Wed, 22 Dec 2010 23:22:40 +0000 (18:22 -0500)]
mke2fs: take the device size into account when determining the size type
If the file system size was not specified on the command line, we were
always using the usage type "floppy" since we didn't determine the
device size until after calling parse_fs_types(). Doh!
Eric Sandeen [Tue, 21 Dec 2010 21:32:05 +0000 (15:32 -0600)]
resize2fs: do not clear resize inode for 0 resvd blocks
I ran into odd behavior where mkfs.ext4 of a 16T filesystem would
create a resize inode with 0 reserved blocks, and mark the resize_inode
feature.
A subsequent slight downward resize of the filesystem would remove
the resize inode, making any further offline resizing impossible.
This is especially odd in light of the fact that a large downward
resize (say, to 8T) will actually add blocks to the resize inode -
so a small resize removes it, a large resize expands it ...
projects / thirdparty / e2fsprogs.git / log
