Darrick J. Wong [Mon, 16 Dec 2013 04:50:03 +0000 (23:50 -0500)]
libext2fs: only punch complete clusters
When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free
any block in a cluster has the effect of freeing the entire cluster.
This is problematic if a caller instructs us to punch, say, blocks
12-15 of a 16-block cluster, because blocks 0-11 now point to a "free"
cluster.
The naive way to solve this problem is to see if any of the other
blocks in this logical cluster map to a physical cluster. If so, then
we know that the cluster is still in use and it mustn't be freed.
Otherwise, we are punching the last mapped block in this cluster, so
we can free the cluster.
The implementation given only does the rigorous checks for the partial
clusters at the beginning and end of the punching range.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 16 Dec 2013 04:26:38 +0000 (23:26 -0500)]
ext2fs,blkid: delete unused LIST_HEAD() macro
The LIST_HEAD() macro conflicts with the <sys/queue.h> declaration
of the same name. Delete the unused LIST_HEAD() macro from the
libext2fs and libblkid headers to avoid compiler warnings. It can
be replaced by INIT_LIST_HEAD() or LIST_HEAD_INIT() if needed.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 16 Dec 2013 04:26:25 +0000 (23:26 -0500)]
ext2fs: quiet compiler warnings
Include ext2fsP.h in fileio.c for ext2fs_file_block_offset_too_big()
declaration. Fix up the declaration to mark it extern in the header.
Include <strings.h> header for strcasecmp() in tune2fs.c if available,
as described in the strcasecmp(3) man page, instead of doing this
indirectly by declaring _BSD_SOURCE and getting it from <string.h>.
If CONFIG_QUOTA is undefined, parse_quota_opts() is unused in
tune2fs.c so #ifdef it out.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 16 Dec 2013 03:12:43 +0000 (22:12 -0500)]
debugfs: clean up debugfs man page and help text
Clean up the formatting of the debugfs(8) man page:
- add command aliases to the main descriptions
- use bold for sub-commands and italics for their arguments
- remove trailing spaces
- add "dirty" sub-command
- clarify "block_dump" and "zap_block" sub-commands
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 16 Dec 2013 03:11:40 +0000 (22:11 -0500)]
build: quiet LLVM non-literal string format warning
Compiling with LLVM generates a large number of warnings due
to the use of _() for wrapping strings for i18n:
warning: format string is not a string literal
(potentially insecure) [-Wformat-security]
./nls-enable.h:4:14: note: expanded from macro '_'
#define _(a) (gettext (a))
^~~~~~~~~~~~
These warnings are fixed by using "%s" as the format string,
and then _() is used as the string argument.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Mon, 16 Dec 2013 03:09:24 +0000 (22:09 -0500)]
build: use long long for __u64 by default
Don't print a verbose configure error in parse-types.h if
<asm/types.h> missing and __[SU]*_TYPEDEF is unset. This is
always the case for non-Linux builds.
The printf formatting strings all use "%llu" for printing 64-bit
values and this it produces a large number of warnings if __u64
is defined as "unsigned long". If __U64_TYPEDEF isn't set use
"unsigned long long" for __u64 in ext2-types.h and blkid-types.h
by default instead of using "unsigned long".
Fix a few places where "%d" or "%u" or "%Lu" were used to print a
64-bit value, by converting them to use "%lld" or "%llu" instead.
Fix a few places where "%lu" was used to print .tv_usec, by casting
the variable to "(long)" since .tv_usec is "int" on some systems.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Sun, 15 Dec 2013 01:51:04 +0000 (20:51 -0500)]
tune2fs: forbid changing uuid on an uninit_bg filesystem
The old uninit_bg checksums depend on the UUID, so prohibit changes to
the UUID if a checksumming filesystem is mounted, because this
introduces a nasty race where the kernel and tune2fs are both trying
to rewrite group descriptors at the same time, with different ideas
about what the UUID is.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Sun, 15 Dec 2013 00:46:50 +0000 (19:46 -0500)]
libext2fs: clamp block-map punch range end to 2^32 blocks
In the ^extent case, passing ~0ULL as the 'end' parameter to
ext2fs_punch() causes the (end - start + 1) calculation to overflow to
zero. Since the old-style mapped block files cannot have more than
2^32 blocks, just clamp it to ~0U.
This fixes a regression in t_quota_2off with the patch "libext2fs: use
ext2fs_punch() to truncate quota file" applied.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 18:31:35 +0000 (13:31 -0500)]
libext2fs: zero block contents past EOF when setting size
When we set the file size, find the block containing EOF, and zero
everything in that block past EOF so that we can't return stale data
if we ever use fallocate or truncate to lengthen the file.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Wed, 11 Dec 2013 01:22:07 +0000 (17:22 -0800)]
e2fsck: in rehash, mark newly allocated extent blocks as found
When we're rehashing directories, it's possible that an extent block
(or a map block) could be (silently) allocated by the underlying
libext2fs when expanding the directory. This silent allocation is not
captured in block_found_map, which is disastrous if later the rehash
process expands another directory and uses that same block from
before without realizing that it's now in use.
Therefore, if we notice that the free block count has dropped by more
than what e2fsck allocated itself during the expansion, we iterate the
directory's blocks a second time to ensure that these silent
allocations are marked in the found blocks bitmap.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 18:25:40 +0000 (13:25 -0500)]
libext2fs: don't error out when punching a totally sparse file
If we're asked to punch a file with no data blocks mapped to it and a
non-zero length, we don't need to do any work in ext2fs_punch_extent()
and can return success. Unfortunately, the extent_get() function
returns "no current node" because it (correctly) failed to find any
extents, which is bubbled up to callers. Since no extents being found
is not an error in this corner case, fix up ext2fs_punch_extent() to
return 0 to callers.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 18:23:51 +0000 (13:23 -0500)]
libext2fs: fix punching extents when there are no left extents
When deleting an entire extent, we cannot always slip to the previous
leaf extent because there might not /be/ a previous extent.
Attempting to correct for that error by asking for the 'current' leaf
extent also doesn't work, because the failed attempt to change to the
previous extent leaves us with no current extent.
Fix this problem by recording the lblk of the next extent before
deleting the current extent and _goto()ing to the next extent after
the deletion.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 18:19:46 +0000 (13:19 -0500)]
libext2fs: fail fileio write if we can't allocate a block
If we're using ext2fs_file_write() to write to a hole in a file,
ensure that we can actually allocate the block before updating i_size.
In other words, don't update i_size and don't return success if we hit
an error while allocating space.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 18:08:45 +0000 (13:08 -0500)]
e4defrag: defensively check results of sysconf(_SC_PAGESIZE)
sysconf(_SC_PAGESIZE) will probably never return an error, but just in
case it does, we shouldn't pass what looks like a huge number to
sync_file_range() and posix_fadvise().
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 18:07:33 +0000 (13:07 -0500)]
e4defrag: don't crash if umounts the filesystem races with us
If someone umounts the filesystem between statfs64() and the getmntent()
iteration, we can exit the loop having never set mnt_type, and strcmp
can crash. Fix the potential NULL dereference.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 17:49:14 +0000 (12:49 -0500)]
debugfs: handle 64bit block numbers
debugfs should use strtoull wrappers for reading block numbers from
the command line. "unsigned long" isn't wide enough to handle block
numbers on 32bit platforms.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 17:48:33 +0000 (12:48 -0500)]
libext2fs: make symlinks safe for 64bit blocks and extents
If we have to create a big symlink (i.e. one that doesn't fit into
i_block[]), we are not 64bit block safe and the namei code does not
handle extents at all. Fix both.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 17:43:36 +0000 (12:43 -0500)]
debugfs: fix various minor bogosity
We should really use the ext2fs memory allocator functions in
copy_file(), and we really should return a value if there's allocation
problems.
Also fix up a minor bogosity in an error message.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com> Cc: Robert Yang <liezhi.yang@windriver.com> Cc: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 17:40:31 +0000 (12:40 -0500)]
libext2fs: don't allow ridiculously large logical block numbers
Forbid clients from trying to map logical block numbers that are
larger than the lblk->pblk data structures are capable of handling.
While we're at it, don't let clients set the file size to a number
that's beyond what can be mapped.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 17:29:07 +0000 (12:29 -0500)]
debugfs: fix init_filesys help text
The help text for debugfs' init_filesys command is incorrect; the
second parameter is the size of the filesystem in blocks, not the size
of an individual filesystem block. There is in fact no way to set
that parameter.
Reported-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Wed, 11 Dec 2013 01:18:37 +0000 (17:18 -0800)]
mke2fs: load configfile blocksize setting before 64bit checks
mke2fs has a series of checks to ensure that we don't create a
filesystem too big for its blocksize -- if auto-64bit is on, then it
turns on 64bit; otherwise it complains. Unfortunately, it performs
these checks before looking in mke2fs.conf for a blocksize, which
means that the checks are incorrect if the user specifies a non-4096
blocksize in the config file and says nothing on the command line.
The bug also has the effect of mandating a 4k block size on any block
device larger than 4T in that situation. Therefore, read the block
size from the config file before performing the 64bit checks.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Wed, 11 Dec 2013 01:18:27 +0000 (17:18 -0800)]
libext2fs: fix tests that set LARGE_FILE
For each site where we test for a large file (> 2GB) and set the
LARGE_FILE feature, use a helper function to make the size test
consistent with the test that's in e2fsck. This fixes the fsck
complaints when we try to create a 2GB journal (not so hard with 64k
block size) and fixes the incorrect test in fileio.c.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Thu, 12 Dec 2013 17:06:03 +0000 (12:06 -0500)]
libext2fs: don't overflow when punching indirect blocks with large blocks
On a FS with a rather large blockize (> 4K), the old block map
structure can construct a fat enough "tree" (or whatever we call that
lopsided thing) that (at least in theory) one could create mappings
for logical blocks higher than 32 bits. In practice this doesn't
happen, but the 'max' and 'iter' variables that the punch helpers use
will overflow because the BLOCK_SIZE_BITS shifts are too large to fit
a 32-bit variable. The current variable declarations also cause punch
to fail on TIND-mapped blocks even if the file is < 16T. So enlarge
the fields to fit.
Yes, this is an obscure corner case, but it seems a little silly if we
can't punch a file's block 300,000,000 on a 64k-block filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Lev Solomonov [Thu, 5 Dec 2013 02:55:22 +0000 (21:55 -0500)]
debugfs: fix range mode of freei
commit 2ae5d1fdb82b31bf270c4699c59a087c846db2f0 was supposed to teach
'seti' and 'freei' to act on a range of consecutive inodes. apparently
only 'seti' has learned: 'freei' doesn't advance the ino, repeatedly
acting on the same one instead.
Signed-off-by: Lev Solomonov <solo@primarydata.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Wed, 4 Dec 2013 01:02:12 +0000 (20:02 -0500)]
mmp: fix 64-bit handling of s_mmp_block
Fix the checking of s_mmp_block in e2fsck_pass1() and
ext2fs_mmp_read() to handle the high 32 bits of s_blocks_count.
Remove redundant check of s_mmp_block in do_dump_mmp() right before
ext2fs_mmp_read() is called.
Also fix s_blocks_count_hi in check_backup_super_block(), since it
cannot use the ext2fs_blocks_count() helper easily.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger [Tue, 3 Dec 2013 05:11:55 +0000 (00:11 -0500)]
tests: check backing filesystem can handle large file
The new resize tests create 2TB test files, but tmpfs in kernels
before 3.1 have a max file size of 256GB. Ext3 may also have
a size limit for smaller blocksize filesystems.
Fix the resize_test script to verify that $TMPFILE can be resized
to the final test size, and if that fails try creating the file on
the local filesystem instead of in $TMPDIR. If that cannot hold
the large filesystem, skip the test.
Signed-off-by: Andreas Dilger <adilger@dilger.ca> Tested-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Whitney [Tue, 3 Dec 2013 04:37:02 +0000 (23:37 -0500)]
libext2fs: fix printf conversion spec in tst_iscan.c
A recent patch to fix blk_t to blk64_t assignment mismatches in
e2fsprogs (commit 4dbfd79d1458ce1259b951377e341aeb6197f8c1) created
a printf conversion spec / argument type mismatch in tst_iscan.c.
Fix this to avoid truncation of the printed value and to silence
a compiler warning seen when "make check" is run.
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 3 Dec 2013 04:07:32 +0000 (23:07 -0500)]
e2fsck: add support for 64-bit extended attribute block refcounting
If we have a 64-bit file system with extended attribute blocks, e2fsck
would not correctly handle EA blocks that were located beyond the
32-bit block number boundary. Fix this by teaching
e2fsck/ea_refcount.c to use 64-bit block numbers.
Theodore Ts'o [Tue, 3 Dec 2013 03:26:58 +0000 (22:26 -0500)]
e2fsck: fix j_maxlen if the file system is exactly 1 << 32 blocks
If the external journal device has exactly 1 << 32 blocks,
journal->j_maxlen would get set to zero, which would cause e2fsck to
declare the journal to be invalid.
Theodore Ts'o [Tue, 3 Dec 2013 02:51:24 +0000 (21:51 -0500)]
dumpe2fs: fix printing of block offsets for 64-bit file systems
Use ext2fs_group_first_block2() instead of ext2fs_group_first_block()
to avoid dumpe2fs from printing crazy block offsets when we have block
numbers which are larger than 32 bits.
Theodore Ts'o [Tue, 3 Dec 2013 02:49:24 +0000 (21:49 -0500)]
e2fsck: use blk_t instead of blk64_t in check_resize_inode()
The resize inode only works on 32-bit block numbers, so use blk_t
instead of blk64_t. This avoids some -Wconversion noise, and slims
the compiled code slightly, especially on 32-bit platforms.
Kit Westneat [Tue, 3 Dec 2013 00:11:52 +0000 (19:11 -0500)]
e2fsck: use ext2fs_write_dir_block3() instead of ext2fs_write_dir_block()
The use of ext2fs_write_dir_block() meant that attempts to fix
deleted/unused inodes in a directory would not be fixed for file
systems with 64-bit block numbers. (And some random block with the
high 32-bits cleared would get corrupted.)
Fix a similar problem when expanding directories and when creating the
lost+found dirctory.
Signed-off-by: Kit Westneat <kwestneat@ddn.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Kit Westneat [Mon, 2 Dec 2013 19:47:08 +0000 (14:47 -0500)]
libext2fs: set the fs block size to new_io in ext2fs_rewrite_to_io()
e2image manually opens a new IO channel, and then sets the file system
to use this new IO channel via ext2fs_rewrite+to_io(). We need to
make sure the IO channel is set to the file system's block size to
avoid some nasty buffer overruns.
[ Modified by tytso to use io_channel_set_blksize() ]
Theodore Ts'o [Mon, 2 Dec 2013 18:14:43 +0000 (13:14 -0500)]
e2image: pass the correct size write_header
Commit bf0449b1a654, which added the ability to write qcow2 files,
generalized the write_header() file to take the size of the header
structure which it writes out. Unfortunately, it changed the call
which supported original e2image format to pass in fs->blocksize,
instead of the actual size of the e2image header structure (which is
substantially smaller than fs->blocksize). This meant that we copied
in stack garbage into the e2image file, and it made valgrind quite
unhappy.
Theodore Ts'o [Thu, 24 Oct 2013 00:09:12 +0000 (20:09 -0400)]
Fix optional static code analysis using sparse
Commit 832cb612: "e2fsprogs: add (optional) sparse checking to the
build" breaks systems that are not using GNU make. In addition, it
breaks if the developer tries to build in a subdirectory (i.e., if he
or she tries running "make" in the misc or e2fsck or lib/ext2fs
directory), since CHECK_CMD is not set.
Fix this by moving the sparse setup to MCONFIG.
Cc: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 18 Oct 2013 04:49:16 +0000 (21:49 -0700)]
mke2fs: don't let resize= turn on resize_inode when meta_bg is set
Passing the "-E resize=NNN" option to mke2fs sets the resize_inode
feature. However, resize_inode and meta_bg are mutually exclusive;
unfortunately, we check this constraint before we parse the extended
options. Fix this by moving this check after the calls
parse_extended_opts().
Reported-by: "Darrick J. Wong" <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Wed, 23 Oct 2013 16:51:32 +0000 (12:51 -0400)]
Fix noquota build
We need to build libquota even if the quota code is disabled. This
fixes a build regression introduced by commit 43075b42bd: 'quota: fix
disabling quota, add quota tests".
Phillip Susi [Wed, 16 Oct 2013 03:37:51 +0000 (23:37 -0400)]
mke2fs: clarify inode ratio in man page
The man page still said it was not possible to change the number of
inodes on a filesystem after creating it. You actually can by
resizing the fs, so clarify this language a bit.
Robert Yang [Mon, 14 Oct 2013 02:09:12 +0000 (22:09 -0400)]
contrib/populate-extfs.sh: use debugfs to populate extX fs
This script uses debugfs command to populate the ext2/3/4 filesystem
from a given directory, it is a little similar to genext2fs, but this
one fully supports ext3 and ext4.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Acked-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Robert Yang [Mon, 14 Oct 2013 02:08:40 +0000 (22:08 -0400)]
debugfs.c: do sparse copy when src is a sparse file
Let debugfs do sparse copy when src is a sparse file, just like
"cp --sparse=auto"
* For the:
#define IO_BUFSIZE 64*1024
this is a suggested value from gnu coreutils:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/ioblksize.h;h=1ae93255e7d0ccf0855208c7ae5888209997bf16;hb=HEAD
* Use malloc() to allocate memory for the buffer since put 64K (or
more) on the stack seems not a good idea.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Acked-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Robert Yang [Mon, 14 Oct 2013 02:07:57 +0000 (22:07 -0400)]
debugfs.c: the max length of debugfs argument is too short
The max length of debugfs argument is 256 which is too short, the
arguments are two paths, the PATH_MAX is 4096 according to
/usr/include/linux/limits.h, so use BUFSIZ (which is 8192 on Linux
systems), that's also what the ss library uses.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Acked-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 12 Oct 2013 03:02:12 +0000 (23:02 -0400)]
e2fsprogs: add (optional) sparse checking to the build
Run sparse against source files when building e2fsprogs with 'make C=1'. If
instead C=2, it configures basic ext2 types for bitwise checking with sparse,
which can help find the (many many) spots where conversion errors are
(possibly) happening.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Sat, 12 Oct 2013 01:20:54 +0000 (21:20 -0400)]
libext2fs: openfs() musn't allow bigalloc without EXT2_FLAGS_64BITS
Currently, only the new 64-bit bitmap implementation supports the
block<->cluster conversions that bigalloc requires. Therefore, if we
have a bigalloc filesystem, require EXT2_FLAGS_64BITS be passed in to
ext2fs_open(). This does not mean that bigalloc file systems have to
be 64-bits; just that the userspace utilities have to be able to use
the new 64-bit capable library functions.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Darrick J. Wong [Sat, 12 Oct 2013 01:20:22 +0000 (21:20 -0400)]
mke2fs: complain about creating 64bit filesystems without extents
A 64bit filesystem without extents is not terribly useful, because the
old block map format does not support pointing to high block numbers.
Warn the user who tries to create such an animal.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Tue, 8 Oct 2013 16:08:46 +0000 (12:08 -0400)]
libext2fs: search all possible blocks in implied_cluster_alloc()
implied_cluster_alloc() is written such that if the the user passes in
a logical block that is the zeroth block in a logical cluster (lblk %
cluster_ratio == 0), then it will assume that there is no physical
cluster mapped to any other part of the logical cluster.
This is not true if we happen to be allocating logical blocks in
reverse order. Therefore, search the whole cluster, except for the
lblk that we passed in.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Tue, 8 Oct 2013 15:51:23 +0000 (11:51 -0400)]
libext2fs: fix off-by-one error in file truncation
When told to truncate a file, ext2fs_file_set_size2() should start with
the first block past the end of the file. The current calculation
jumps one more block ahead, with the result that it fails to hack off
the last block. Adding blocksize-1 and dividing is sufficient to find
the last block.
Reviewed-by: Lukas Czerner <lczerner@redhat.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Niu Yawei [Tue, 8 Oct 2013 03:30:10 +0000 (11:30 +0800)]
libext2fs: update i_size in ext2fs_file_write()
ext2fs_file_write() needs to update i_size on successful write,
otherwise, ext2fs_file_read() in same open/close cycle will not
be able to read the just written data.
This fixes a bug which results in the the problem of quotacheck
triggered on 'tune2fs -O quota' failed to write back multiple
users/groups accounting information.
Darrick J. Wong [Mon, 7 Oct 2013 13:57:36 +0000 (09:57 -0400)]
e2fsprogs: fix inode and block relocation functions to use blk64_t
The inode and block relocation functions aren't currently compiled in
(so we don't need to worry about breaking ABI compatibility). They
were originally intended for use by resize2fs, but we never ended up
using them, so (wisely) they weren't ever included in libext2fs as an
exported interface (they're not even compiled by the Makefile).
Fix them so that in case we ever use them, so that in places where raw
data types (int, long, etc.) stood in for blk_t and blk64_t. Also fix
some sites where we should probably be using blk64_t.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix all the places where we should be using a blk64_t instead of a
blk_t. These fixes are more severe because 64bit values could be
truncated silently.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 7 Oct 2013 13:51:35 +0000 (09:51 -0400)]
libext2fs: ind_punch() must not stop examining blocks prematurely
When we're iterating the main loop in ind_punch(), "offset" tracks how
far we've progressed into the block map, "start" tells us where to
start punching, and "count" tells us how many blocks we are to punch
after "start". Therefore, we would like to break out of the loop once
the "offset" that we're looking at has progressed past the end of the
punch range. Unfortunately, if start !=0, the if-break clause in the
loop causes us to break out of the loop early.
Therefore, change the breakout test to terminate the loop at the
correct time.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 7 Oct 2013 13:51:20 +0000 (09:51 -0400)]
libext2fs: allow callers to punch a single block
The range of blocks to punch is treated as an inclusive range on both
ends, i.e. if start=1 and end=2, both blocks 1 and 2 are punched out.
Thus, start == end means that the caller wishes to punch a single
block. Remove the check that prevents us from punching a single
block.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 7 Oct 2013 13:35:22 +0000 (09:35 -0400)]
libext2fs: rewind extent pointer when totally deleting an extent
During a punch operation, if we decide to delete an extent out of the
extent tree, the subsequent extents are moved on top of the current
extent (that is to say, they're memmmove'd down one slot). Therefore
it is not correct to advance to the next leaf because that means we
miss half the extents in the range! Rereading the current pointer
should be fine.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Tue, 1 Oct 2013 01:27:28 +0000 (18:27 -0700)]
libext2fs: set the large_file feature flag when setting i_size > 2GB
If someone tries to write a file that is larger than 2GB, we need to
set the large_file feature flag to affirm that i_size_hi can hold
meaningful contents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 1 Oct 2013 01:26:55 +0000 (18:26 -0700)]
libext2fs: only link an inode into a directory once
The ext2fs_link helper function link_proc does not check the value of
ls->done, which means that if the function finds multiple empty spaces
that will fit the new directory entry, it will create a directory
entry in each of the spaces. Instead of doing that, check the done
value and don't do anything more if we've already added the directory
entry.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 1 Oct 2013 03:07:27 +0000 (23:07 -0400)]
tests: add test for resize2fs -M with inode table in middle of block group
Eric Sandeen reported that Fedora's mke2fs when compiled for ppc was
creating a file system which caused problems with resize2fs -M.
Closer examination showed that the problem was file system which
looked like this:
Group 0: (Blocks 1-1024)
Primary superblock at 1, Group descriptors at 2-2
Block bitmap at 66 (+65), Inode bitmap at 67 (+66)
Inode table at 68-99 (+67)
Group 1: (Blocks 1025-1246)
Backup superblock at 1025, Group descriptors at 1026-1026
Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66)
Inode table at 1092-1123 (+67)
It's not obvious to me why Fedora's ppc mke2fs is creating file
systems like this (I can't reproduce this on debian ppc systems), but
resize2fs -M should be able to deal with such file systems, which is
what this test is designed to check.
Theodore Ts'o [Tue, 1 Oct 2013 02:55:21 +0000 (22:55 -0400)]
resize2fs: relocate inode table blocks if necessary when shrinking
If the file system is being shrunk, and a block group's inode table
falls beyond the end of the inode table, we need to try to relocate
the inode table blocks.
Theodore Ts'o [Tue, 1 Oct 2013 02:35:14 +0000 (22:35 -0400)]
resize2fs: fix -M size calculations to avoid cutting off the inode table
If the file system's inode table blocks in the last block group are
located in the middle or the end of the block group, it's possible for
resize2fs -M to use a size which will require relocating the inode
table blocks in the last block group. This can lead to all sorts of
problems, so solve it by simply guaranteeing that we will never do
that.
Reported-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Eric Whitney [Fri, 27 Sep 2013 00:17:09 +0000 (20:17 -0400)]
tests: add another test for uninit extents past eof
Commit d3f32c2db8 was intended to detect extents found outside their
proper location in the extent tree, including invalid extents at the
end of an extent block. However, it incorrectly reported legal
uninitialized extents created by fallocate() at the end of file with
the FALLOC_FL_KEEP_SIZE flag as false positives. xfstests
generic/263 (among others) caught this problem, while the e2fsprogs
test f_uninit_ext_past_eof did not. The latter test failed to
detect the problem in part because it uses a test file whose i_size
is 0.
Add a test derived from the fsx-based test case in xfstests
generic/263 consisting of a file with non-zero length, more than
four extents total, and two uninitialized extents past EOF to
reliably reproduce commit d3f32c2db8's false positive behavior.
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Whitney [Mon, 9 Sep 2013 14:53:03 +0000 (10:53 -0400)]
e2fsck: don't report uninit extents past EOF invalid
Commit d3f32c2db8 introduced a regression that caused e2fsck failures
in xfstests generic 013, 070, 083, 091, and 263. Uninitialized
extents created by fallocate() at the end of file with the
FALLOC_FL_KEEP_SIZE flag were identified as invalid. However,
because the file size is not increased when FALLOC_FL_KEEP_SIZE is
used, uninitialized extents can correctly contain blocks located past
the end of file.
Fix this by filtering out possible invalid extents if they are
uninitialized and extend past the block containing the end of file.
Signed-off-by: Eric Whitney <enwlinux@gmail.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Zheng Liu [Mon, 5 Aug 2013 05:02:43 +0000 (13:02 +0800)]
libext2fs: fix a coding style for EXT2_NO_MTAB_FILE
When we define an error in lib/ext2fs/ext2_err.et.in, we will always use
EXT2_ET_* prefix for a new error. But EXT2_NO_MTAB_FILE doesn't obey
this rule. So fix it.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
and we resize the fs in such a way that all of those blocks must
be moved down, we do them one at a time. Eventually we move 1-block
extent A) to a lower block, and then follow it with the other
blocks in the next logical offsets from extent C) in the next
interior node B).
The userspace extent code tries to merge, so when it finds that
logical 13228 can be merged with logical 13227 into a single extent,
it does. And so on, all through extent C), up to block 13250 (why
not 13251? [1]), and eventually move the node block as well.
So we end up with this when all the blocks are moved post-resize:
All those adjacent blocks got moved into extent D), which is nice -
but the next interior node E) was never updated to reflect its new
starting point - it says the leaf extents beneath it start at 13228,
when in fact they start at 13251.
So as we move blocks one by one out of original extent C) above, we
need to keep updating C)'s parent node B) for a proper starting point.
fix_parents() does this.
Once the tree is corrupted like this, more corruption can
ensue post-resize, because we traverse the tree by interior nodes,
relying on their start block to know where we are in the tree.
If it gets off, we'll end up inserting blocks into the wrong part
of the tree, etc.
I have a testcase using fsx to create a complex extent tree which
is then moved during resize; it hit this corruption quite easily,
and with this fix, it succeeds.
Note the first hunk in the commit is for going the other way,
moving the last block of an extent to the extent after it; this
needs the same sort of fix-up, although I haven't seen it in
practice.
[1] We leave the last block because a single-block extent is its
own case, and there is no merging code in that case. \o/
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 9 Sep 2013 14:40:36 +0000 (10:40 -0400)]
resize2fs: use blk64_t and location getters for free_gdp_blocks()
free_gdp_blocks needs to be taught to use 64-bit fields and the appropriate
getters, otherwise it'll truncate high block numbers (when, say, resizing a
>16T fs) and mark the low numbered group descriptor blocks as free. Yikes.
Reported-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 9 Sep 2013 14:39:29 +0000 (10:39 -0400)]
libext2fs: ext2fs_dup_handle should not alias MMP buffers
It turns out that resize2fs uses ext2fs_dup_handle to duplicate fs handles. If
MMP is enabled, this causes both handles to share MMP buffers, which is bad
news when it comes time to free both handles. Change the code to (we hope) fix
this. This prevents resize2fs from failing with a double-free error when
handed a MMP filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Eric Sandeen [Mon, 9 Sep 2013 14:33:20 +0000 (10:33 -0400)]
e2fsck: don't try to stop mmp if there is no superblock set up
Under some failure cases, we can get to fatal_error()
without even having a superblock set up. In that case,
ext2fs_mmp_stop() will segfault when it tries to dereference
fs->super.
Check for the existence of a superblock before we go
down the ext2fs_mmp_stop() path to avoid this problem.
projects / thirdparty / e2fsprogs.git / log
