Theodore Ts'o [Fri, 10 Jun 2011 22:51:58 +0000 (18:51 -0400)]
libext2fs: teach block allocation functions about bigalloc/clusters
Optimize ext2fs_new_block2() and ext2fs_get_free_blocks2() when
bigalloc is enabled. Also fix the uninitialized block bitmap code so
that it correctly deals clustered allocation.
Theodore Ts'o [Fri, 10 Jun 2011 21:55:09 +0000 (17:55 -0400)]
libext2fs: teach bitmap functions about bigalloc/cluster
This patch makes the following changes:
* ext2fs_allocate_block_bitmap() now allocates a bitmap with cluster
granularity for bigalloc file systems. For mke2fs and e2fsck, a
newly added function, ext2fs_allocate_subcluster_bitmap() allocates
a bitmap with block granularity (even for bigalloc file systems).
The newly added function ext2fs_get_bitmap_granularity() will return
the number of bits (log2) of the granularity used by the bitmap.
* The ext2fs_{mark,unmark,test}_block_bitmap2() functions will shift
their passed-in argument by log2(cluster_ganularity) bits right.
This means that the arguments for the single-argument bitmap
functions will be interpreted with block granluarity, since this
minimizes code changes in the rest of the code base.
* The ext2fs_{get,set}_block_bitmap_range() functions will interpret
their arguments in cluster granularity. This is a bit inconsistent,
but the caller of those functions will need to be taught about the
subtleties of clusters for bigalloc file systems.
Andreas Dilger [Tue, 7 Jun 2011 16:22:29 +0000 (10:22 -0600)]
mke2fs: Don't erase flash device if "-n" is given
If "mke2fs -n" is used, there should be no changes to the underlying
device. Unfortunately, when the "discard" option was added in commit c7cd908be59f48c66b4f3ac9a631ffe3dde4f1ab, it did not check for the "-n"
flag, and will discard all data on a flash device even if "-n" is given.
Check for the "noaction" flag before discarding any filesystem data.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 4 Jun 2011 14:20:47 +0000 (10:20 -0400)]
libext2fs: change fs->clustersize to fs->cluster_ratio_bits
The log2 of the ratio of cluster size to block size is far more useful
than just storing the cluster size. So make this change, and then
define basic utility macros: EXT2FS_CLUSTER_RATIO(),
EXT2FS_CLUSTER_MASK(), EXT2FS_B2C(), EXT2FS_C2B(), and
EXT2FS_NUM_B2C().
Theodore Ts'o [Sat, 4 Jun 2011 20:40:26 +0000 (16:40 -0400)]
libext2fs: change EXT2_MAX_BLOCKS_PER_GROUP() to be cluster size aware
Change the EXT2_MAX_BLOCKS_PER_GROUP so that it takes the cluster size
into account. This way we can open bigalloc file systems without
ext2fs_open() thinking that they are corrupt.
Theodore Ts'o [Sat, 4 Jun 2011 20:36:19 +0000 (16:36 -0400)]
libext2fs: require cluster size == block_size when opening a !bigalloc fs
In ext2fs_open() check to make sure the cluster size superblock field
is the same as the block size field when the bigalloc feature is not
set. This is necessary since we will start introducing calculations
based on the cluster size field.
Eric Sandeen [Tue, 31 May 2011 23:59:56 +0000 (19:59 -0400)]
e2fsck: don't check/clone duplicate xattr blocks in fs without xattr feature
I had an extremely corrupted customer filesystem which, after thousands
of lines of e2fsck output, found one more problem on an immediately
subsequent e2fsck. In short, a file had had its i_file_acl block
cloned due to being a duplicate. That ultimately got cleared
because the fs did not have the xattr feature, and the inode
was subsequently removed due to invalid mode.
The 2nd e2fsck pass found the cloned xattr block as in use, but
not owned by any file, and had to fix up the block bitmaps.
Simply skipping the processing of duplicate xattr blocks on a
non-xattr filesystem seems reasonable, since they will be cleared
later in any case.
(also fix existing brace misalignment)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2fsprogs: Unify the upper limit of reserved blocks count
In e2fsprogs, the upper limit of reserved blocks count is a half of
filesystem's blocks count. This patch fixes the incorrect checks of
reserved blocks count.
Eric Sandeen [Mon, 4 Apr 2011 19:11:52 +0000 (15:11 -0400)]
mke2fs: don't set stripe/stride to 1 block
Block devices may set minimum or optimal IO hints equal to
blocksize; in this case there is really nothing for ext4
to do with this information (i.e. search for a block-aligned
allocation?) so don't set fs geometry with single-block
values.
Zeev also reported that with a block-sized stripe, the
ext4 allocator spends time spinning in ext4_mb_scan_aligned(),
oddly enough.
Lukas Czerner [Wed, 18 May 2011 11:36:55 +0000 (13:36 +0200)]
tests: New i_e2image test to validate image creation/conversion
This commit adds new regression test called i_e2image which should
validate expected behaviour of e2image raw and qcow2 image creation
and qcow2 -> raw image conversion. You can run it with "make check" as
the rest of regression tests.
Testing is performed on three filesystem images with different block
sizes (1024, 2048, 4096). It creates raw and qcow2 images from the
original and then convert qcow2 image back to raw image. The results are
checksummed and compared with pre-prepared results. All md5sums should
stay the same and raw image created from original and qcow2 image should
be the same as well (just for the record).
Lukas Czerner [Wed, 18 May 2011 12:20:47 +0000 (14:20 +0200)]
e2image: Support for conversion QCOW2 image into raw
This commit adds support for converting QCOW2 image created previously
with e2image into raw image. The QCOW2 image is detected automatically,
so there is not new option. Just use following command:
e2image -r image.qcow image.raw
No that this tool is aimed to quickly convert qcow2 image created with
e2image into raw image. In order to improve speed we are doing some
assumption I believe might not be true for regular qcow2 images. So it
was not tested with regular QCOW2 images and it might not work with
them. The intention of this tool is only convert images previously
created by e2image.
Note that there is nothing special with QCOW2 images created by e2images
and it can be used with tools like qemu-img, or qemu-nbd without any
problems.
Lukas Czerner [Wed, 18 May 2011 11:36:53 +0000 (13:36 +0200)]
e2image: Add support for qcow2 format
This commit adds support for exporting filesystem into QCOW2 image
format. Like sparse format this saves space, by writing only necessary
(metadata blocks) into image. Unlike sparse image, QCOW2 image is NOT
sparse, hence does not change its size by copying with not-sparse-aware
tools.
New options '-Q' has been added to tell the e2image to use QCOW2 as an
output image format. QCOW2 supports encryption and compression, however
e2image so far does no support such features, however you can still
scramble filenames with '-s' option.
Lukas Czerner [Wed, 18 May 2011 12:19:52 +0000 (14:19 +0200)]
e2fsprogs: Add memory allocation and zero-out helpers
Add functions ext2fs_get_memzero() which will malloc() the memory
using ext2fs_get_mem(), but it will zero the allocated memory afterwards
with memset().
Add function ext2fs_get_arrayzero() which will use calloc() for
allocating and zero-out the array.
Aditya Kali [Tue, 10 May 2011 21:51:31 +0000 (14:51 -0700)]
mke2fs: Allow specifying reserved_ratio via mke2fs.conf
This patch adds support for specifying 'reserved_ratio' (percent blocks
reserved for super user, same as '-m' command line option) in mke2fs.conf.
It adds profile_get_double function in profile.c that allows reading
floating point values from profile files.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
It will cause segfault, but it is caused by another issue. See my
patch "mke2fs: Avoid segmentation fault in
ext2fs_alloc_generic_bmap". And with that patch, the mkfs.ext4 will
fail with the error: /dev/sdb3: Memory allocation failed while setting
up superblock
The reason is that in ext2fs_initialize, we align s_inodes_per_group
to 8, but fails to consider the case that s_inodes_per_group < 8.
So make at least 8 inodes for s_inodes_per_group.
Signed-off-by: Tao Ma <boyu.mt@taobao.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lukas Czerner [Mon, 21 Mar 2011 10:14:00 +0000 (11:14 +0100)]
e2fsprogs: fix memory leak in ext2fs_free_generic_bmap()
In ext2fs_free_generic_bmap() when we are freeing 64-bit bitmap, we do
call free_bmap() to free backend specific bitmap structures, however we
should also free ext2fs_generic_bitmap structure as well.
Lukas Czerner [Mon, 21 Mar 2011 10:13:41 +0000 (11:13 +0100)]
e2fsprogs: fix freeing bitmap in allocation error path
In ext2fs_alloc_generic_bmap() error path, when new bitmap allocation
fails ext2fs_generic_bitmap should be freed, however in current state it
first frees ext2fs_generic_bitmap and then
ext2fs_generic_bitmap->description which is wrong. This commit fix the
free ordering.
Theodore Ts'o [Sun, 8 May 2011 03:14:06 +0000 (23:14 -0400)]
e2fsck: make the "fs is mounted; continue?" prompt more paranoid
A user received the "file system is mounted; do you really want to
continue" prompt, and then instead of typing "n" for no, forgot that
he hadn't declined to continuation question, and typed the up-arrow
key, which in his locale, the 'A' in "^[[A" was interpreted as "yes",
and he lost data.
This was clearly the user's fault, but to make e2fsck a bit safer
against user stupidity/carelessness, we will change the "fs is
mounted; continue?" prompt to default to no, and treat the escape
character (along with the return and space characters, currently) as a
request for the default answer.
Eric Sandeen [Thu, 5 May 2011 18:21:08 +0000 (13:21 -0500)]
filefrag: count 0 extents properly when verbose
/boot/a: 0 extents found
works properly, but
Filesystem type is: ef53
Filesystem cylinder groups is approximately 61
File size of a is 0 (0 blocks, blocksize 1024)
ext logical physical expected length flags
a: 1 extent found
yields 1 extent when it should be 0.
Fix this up by special-casing no extents returned in verbose
mode; skip printing the header for the columns too, since there
are no columns to print.
Also, in nonverbose mode we can set fm_extent_count to 0
so that FIEMAP will just query the extent count without gathering
details; clarify this with a comment.
Addresses-RedHat-Bugzilla: 653234 Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 18 Mar 2011 18:47:15 +0000 (14:47 -0400)]
add new superblock field: s_overhead_blocks
It turns out that it's very hard to calculate overheads in the face of
clustered allocation (bigalloc). This is because multiple metadata
blocks from different block groups can end up in the same allocation
cluster. Calculating the exact overhead requires O(all block bitmaps)
in memory, or O(number of block groups**2) in time. So we will
calculate this at mkfs time and stash it in the superblock.
Theodore Ts'o [Mon, 28 Feb 2011 01:09:54 +0000 (20:09 -0500)]
mke2fs: If the device supports discard, don't print an error message
Check to see if the device supports discard before starting the
progress bar, and then printing an error about inappropriate ioctl for
device (when creating a file system image to a file, for example).
Also, add a function signature in the ext2_io.h header file for
io_channel_discard() and fix an extra, uneeded argument in mke2fs's
call to that function.
Theodore Ts'o [Sat, 26 Feb 2011 02:43:54 +0000 (21:43 -0500)]
Add basic BIGALLOC support for cluster-based allocation
This adds the superblock fields needed so that dumpe2fs works and the
code points and renames the superblock fields from describing
fragments to clusters.
Aditya Kali [Tue, 15 Feb 2011 22:27:27 +0000 (14:27 -0800)]
e2fsprogs: reserving code points for new ext4 quota feature
This patch adds support for detecting the new 'quota' feature in ext4.
The patch reserves code points for usr and group quota inodes and also
for the feature flag EXT4_FEATURE_RO_COMPAT_QUOTA.
Signed-off-by: Aditya Kali <adityakali@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lukas Czerner [Mon, 24 Jan 2011 19:52:00 +0000 (20:52 +0100)]
mke2fs: Display progress report during the device discard
For some time now we are doing initial discard of the device prior to
filesystem creation. However, there is no feedback for the user and
hence on some devices with slow TRIM implementation it may appear that
mke2fs is stuck.
This commit introduce new function mke2fs_discard_device(), which is a
wrapper for io_channel_discard(). The discard is done in chunks of
2GB, which seems reasonably well for both slow and fast devices, and
discard progress is reported back to the user.
I gave up on doing fancy things like align discard according to
discard_alignment, checking for discard granularity and computing
estimate time. First of all, because it would require either new ioctl
to retrieve those information or use of libudev library, none of it
seems to be worth it. Regarding discard_granularity, I doubt there is
any sane device with discard granularity that big it would affect this.
Eric Sandeen [Thu, 17 Feb 2011 21:56:17 +0000 (15:56 -0600)]
e2fsprogs: enable user namespace xattrs by default
User namespace xattrs are generally useful, and I think extN
is the only filesystem requiring a special mount option to
enable them, when xattrs are otherwise available. So this
change sets that mount option into the defaults, via a
mke2fs.conf option.
Note that if xattrs are config'd off, this will lead to a
mostly-harmless:
EXT4-fs (sdc1): (no)user_xattr options not supported
message at mount time...
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Thu, 17 Feb 2011 21:55:15 +0000 (15:55 -0600)]
e2fsprogs: turn off enforced fsck intervals by default
The forced fsck often comes at unexpected and inopportune moments,
and even enterprise customers are often caught by surprise when
this happens. Because a filesystem with an error condition will
be marked as requiring fsck anyway, I submit that the time-based
and mount-based checks are not particularly useful, and that
administrators can schedule fscks on their own time, or tune2fs
the enforced intervals if they so choose. This patch disables the
intervals by default, and I've added a new mkfs.conf option to
turn on the old behavior of random, unexpected, time-consuming
fscks at boot time. ;)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Wed, 16 Feb 2011 18:01:39 +0000 (12:01 -0600)]
e2fsprogs: create com_err.h link in includedir
After debian bug #192277, debian/rules started making a symlink
to com_err.h in /usr/include. Now I have Fedora bug #550889
for the same issue, and perhaps it's time to make this link
by default, rather than fixing it up in packaging steps?
[ Changed by tytso to remove the explicit -s option; this will default
to creating a hard link by default, which slightly faster. If
people want to use symlinks for all links during the install
process, they can use configure option --enable-symlink-install.
The reason for this change is that some file systems, like AFS,
don't support symlinks, and AFS users complain when they can't build
or install into AFS. So I don't want to use symlinks
unconditionally without a way of switching things back and forth,
and it's easier if we just make all links made during the install
process to be hard links or sym links. ]
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 20 Feb 2011 20:19:47 +0000 (15:19 -0500)]
badblocks: Fix up recover_block handling in badblocks
If there was a bad block for block #0, badblocks would never switch
back testing blocks more efficiently. In addition, we were
double-incrementing the blocks to be tested in the read/write test due
to failure to remove code.
Thanks to Ragnar Kjørstad for pointing these problems out.
Theodore Ts'o [Fri, 18 Feb 2011 03:58:21 +0000 (22:58 -0500)]
badblocks: Only report errors when reading/writing one block at a time
With Direct I/O, the kernel can report 0 bytes read even though the
first block has no errors. So there are any errors, we need try to
read/write blocks one at a time and to get an accurate report.
Kazuya Mio [Sun, 19 Dec 2010 22:01:24 +0000 (22:01 +0000)]
e4defrag: Use libext2fs to get the correct superblock information
Currently, e4defrag always does byte-swapping when it gets superblock
information, so the calculation of the best extents count is not
correct on little endian machine. This doesn't cause data corruption,
but it may confuse users by showing the wrong extent count. To solve
this problem, we use ext2fs_open() instead of get_superblock_info()
that is the original function.
Theodore Ts'o [Wed, 22 Dec 2010 23:22:40 +0000 (18:22 -0500)]
mke2fs: take the device size into account when determining the size type
If the file system size was not specified on the command line, we were
always using the usage type "floppy" since we didn't determine the
device size until after calling parse_fs_types(). Doh!
Eric Sandeen [Tue, 21 Dec 2010 21:32:05 +0000 (15:32 -0600)]
resize2fs: do not clear resize inode for 0 resvd blocks
I ran into odd behavior where mkfs.ext4 of a 16T filesystem would
create a resize inode with 0 reserved blocks, and mark the resize_inode
feature.
A subsequent slight downward resize of the filesystem would remove
the resize inode, making any further offline resizing impossible.
This is especially odd in light of the fact that a large downward
resize (say, to 8T) will actually add blocks to the resize inode -
so a small resize removes it, a large resize expands it ...
Namhyung Kim [Mon, 29 Nov 2010 08:55:12 +0000 (17:55 +0900)]
mke2fs.8.in: add missing "big" and "huge" usage-type description
The commit 493024ea1d74e4cb48aac3a24111f5c8da343e9f ("mke2fs: Fix up the
fs type and feature selection for 64-bit blocks") added 'big' and 'huge'
usage-type but was missing description in man page. Add it.
Signed-off-by: Namhyung Kim <namhyung@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Namhyung Kim [Mon, 20 Dec 2010 15:07:10 +0000 (10:07 -0500)]
libext2fs: fix potential build failure with OMIT_COM_ERR
This fixes following build failure when OMIT_COM_ERR is defined:
lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_clear_generic_bitmap’:
lib/ext2fs/gen_bitmap.c:437: error: invalid storage class for function ‘ext2fs_test_clear_generic_bitmap_range’
lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_get_generic_bitmap_end’:
lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_get_generic_bitmap_start’:
lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_unmark_generic_bitmap’:
lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_mark_generic_bitmap’:
lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_test_generic_bitmap’:
lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
make[2]: *** [gen_bitmap.o] Error 1
make[2]: Leaving directory e2fsprogs/lib/ext2fs'
make[1]: *** [all-libs-recursive] Error 1
make[1]: Leaving directory e2fsprogs'
make: *** [all] Error 2
Signed-off-by: Namhyung Kim <namhyung@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 17 Dec 2010 03:53:34 +0000 (22:53 -0500)]
e4defrag: Fix the overflow in e4defrag with > 2GB files
The fallocate() interface on 32-bit machines is defined to use off_t,
not loff_t (even though the system call interface is 64-bit clean).
This causes e4defrag to fail on files greater than 2GB. Fix this by
trying to use fallocate64(), and using the hard-coded syscall if it
does not exist.
Eric Sandeen [Thu, 16 Dec 2010 04:37:34 +0000 (22:37 -0600)]
resize2fs: handle exactly-16T filesystems in resize2fs
Before we go whole-hog on 64-bit e2fsprogs, I wonder if this
is worth considering as a last-minute addition to the 1.41
stream. Currently, mke2fs will shave a block off an exactly-16T
device to fit*, but resize2fs does not do the same, leading
to some asymmetry. This patch fixes that up, and allows 16T
devices to be handled more gracefully in offline resize.
(in fact resize2fs will not even open a 16T device, today).
Eric Sandeen [Tue, 14 Dec 2010 19:00:01 +0000 (13:00 -0600)]
e2fsprogs: fix type-punning warnings
Flags used during RHEL/Fedora builds lead to a couple type-punning
warnings:
recovery.c: In function 'do_one_pass':
recovery.c:539: warning: dereferencing type-punned pointer will break strict-aliasing rules
./csum.c: In function 'print_csum':
./csum.c:170: warning: dereferencing type-punned pointer will break strict-aliasing rules
The two changes below fix this up.
Note that the csum test binary output changes slightly, but this does
not break any tests.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Kazuya Mio [Mon, 13 Dec 2010 14:59:06 +0000 (09:59 -0500)]
e4defrag: fix file blocks calculation
e4defrag uses st_blocks (struct stat) to calculate file blocks. However,
st_blocks also has meta data blocks in addition to file blocks. So, we
calculate file blocks by sum of the extent length.
projects / thirdparty / e2fsprogs.git / log
