Andreas Dilger [Sun, 5 May 2019 22:33:46 +0000 (18:33 -0400)]
mke2fs: fix check for absurdly large devices
The check in mke2fs is intended to be for the number of blocks in the
filesystem exceeding the maximum number of addressable blocks in 2^32
bitmaps, which is (2^32 * 8 bits/byte * blocksize) = 2^47 blocks,
or 2^59 bytes = 512PiB for the common 4KiB blocksize.
However, s_log_blocksize holds log2(blocksize_in_kb), so the current
calculation is a factor of 2^10 too small. This caused mke2fs to fail
while trying to format a 900TB filesystem.
Fixes: 101ef2e93c25 ("mke2fs: Avoid crashes / infinite loops for absurdly large devices") Signed-off-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 5 May 2019 20:43:33 +0000 (16:43 -0400)]
e2fsck: check and fix tails of all bitmap blocks
Currently, e2fsck effectively checks only tail of the last inode and
block bitmap in the filesystem. Thus if some previous bitmap has unset
bits it goes unnoticed. Mostly these tail bits in the bitmap are
ignored; however, if blocks_per_group are smaller than 8*blocksize,
the multi-block allocator in the kernel can get confused when the tail
bits are unset and return bogus free extent.
Add support to libext2fs to check these bitmap tails when loading
bitmaps (as that's about the only place which has access to the bitmap
tail bits) and make e2fsck use this functionality to detect buggy bitmap
tails and fix them (by rewriting the bitmaps).
Reported-by: Jan Kara <jack@suse.cz> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Jan Kara <jack@suse.cz>
Theodore Ts'o [Fri, 3 May 2019 20:42:36 +0000 (16:42 -0400)]
libext2fs: move struct ext2fs_nls_table to the private ext2fsP.h header
Callers of libext2fs don't need to use this structure, and this gives
us the ability to change it later on without worrying about
changing public ABI's.
Theodore Ts'o [Fri, 3 May 2019 17:16:29 +0000 (13:16 -0400)]
Rename the feature "fname_encoding" to be "casefold".
Also change mke2fs so that the encoding and encoding flags are
specified in mke2fs.conf in the fs_types and defaults stanzas instead
of the options stanza.
Merge nls_utf8-norm.c and nls_utf8.c. This also allows us to comment
out functions which we don't actually need for e2fsprogs.
Also fix some gcc -Wall complaints, including one which would have
caused utf8_casefold() to misbehave (this was fixed in the kernel, but
not carried back to e2fsprogs).
Eric Biggers [Mon, 29 Apr 2019 00:35:21 +0000 (20:35 -0400)]
debugfs: fix encoding handling in dx_hash command
Fix the following bugs:
1. 'encoding' and 'hash_flags' are not initialized, causing a segfault.
2. 'hash_flags' incorrectly uses a __bitwise type.
3. The optstring doesn't contain "c" or "e", so the -c and -e options
aren't recognized.
4. The code that handles the -e option always returns.
Fixes: ef733f1a97ec ("debugfs: support encoding when printing the file hash") Reviewed-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Sun, 28 Apr 2019 23:42:41 +0000 (19:42 -0400)]
debugfs: avoid ambiguity when printing filenames
The way debugfs escapes filenames is ambiguous because a sequence like
M-A can mean either the byte 'A' + 128 == 0xc1 or the three bytes
{'M', '-', 'A'}. Similarly, ^A can mean either the byte
'A' ^ 0x40 == 0x01 or the two bytes {'^', 'A'}.
Fix this and simplify the code by switching to a simpler strategy where
all bytes < 32, all bytes >= 127, and backslash are encoded with C-style
hex escape sequences. E.g., the byte 0xc1 will now be encoded as \xc1
rather than M-A as it was before, while a filename consisting of the
three bytes {'M', '-', 'A'} will continue to be shown as M-A.
I want to fix this mainly because I want to use debugfs to retrieve raw
encrypted filenames for ciphertext verification tests. But this doesn't
work if the returned filenames are ambiguous.
Fixes: 68a1de3df340 ("debugfs: pretty print encrypted filenames in the ls command") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
ext2fs: always attempt to load nls table when loading the filesystem
fs->encoding is exposed by the library, so we need to at least try to
load it when populating ext2_filsys. Nevertheless, failing to do so
shouldn't be a fatal error, unless the user really needs that
information. Thus, we ignore this failure during open/initialization
and let the user who needs it validate that field before trying to use
it.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 24 Mar 2019 06:36:08 +0000 (02:36 -0400)]
tests: fix f_valid_ea_in_inode to use the ea_feature
This time for real! The test-ea file has three very long xattrs,
description and description2 which are identical and share an inode to
store their value, and description3, which has a different value, and
thus uses another inode for storing its value.
Darrick J. Wong [Tue, 19 Mar 2019 00:17:32 +0000 (17:17 -0700)]
e2scrub_all: refactor device probe loop
Paul Menzel reported that the e2scrub_all reaper service that runs at
startup takes a long time to run, and Ted Ts'o pointed out that we could
do a lot less work by using lvs as the outer loop in the ext4 filesystem
probe function so that we only have to lsblk the lvm devices containing
ext4 filesystems.
Therefore, refactor the loops to put lvs first, which should boost speed
a bit.
[ Made some of the further optimizations suggested by Lukas Czerner. -- TYT ]
Reported-by: Paul Menzel <pmenzel@molgen.mpg.de> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 20 Mar 2019 15:58:58 +0000 (11:58 -0400)]
e2scrub_all: make sure there's enough free space for a snapshot
If there isn't, skip the volume so we don't spam the system
administrator with error messages. It's quite commkon that there is
is zero free space in the volume group.
Theodore Ts'o [Sun, 10 Mar 2019 23:41:11 +0000 (19:41 -0400)]
e2scrub: check to make sure lvm2 is installed
Not all systems will have the lvm2 package installed, so check for
that. Pretty much all systems should have util-linux installed, but
check for that as well.
Of course, if lvm2 is installed we shouldn't find any LVM devices ---
but eventually the Demon Murphy will find a way to make it happen. :-)
Also, set the PATH so we don't have to worry about the script failing
due to /sbin not being in the path.
Dongyang Li [Wed, 6 Mar 2019 22:59:55 +0000 (22:59 +0000)]
Fix autoheader warnings
autoheader complains after 1.44.6 release:
autoheader: warning: missing template: HAVE_BLKID_PROBE_ENABLE_PARTITIONS
autoheader: Use AC_DEFINE([HAVE_BLKID_PROBE_ENABLE_PARTITIONS], [], [Description])
autoheader: warning: missing template: HAVE_BLKID_PROBE_GET_TOPOLOGY
This could fail automatic rpm builds. Fixes: 7154d97 ("Check for the newer blkid functions without adding blkid to @LIBS@") Signed-off-by: Li Dongyang <dongyangli@ddn.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2image: add -b and -B options to specify where to find the superblock
e2image has no ability to use superblock backup to copy metadata.
This feature can be useful if someone wants to make partition
image and fix it using e2fsck utility.
New -b option allows to pass superblock number, like e2fsck utility does.
e2image doesn't change primary superblock and store it as is, so
it can be fixed using e2fsck latter. Option -B allows setting
superblock size.
Yoshitaka Seto [Wed, 16 Jan 2019 08:47:48 +0000 (17:47 +0900)]
AOSP: misc: use scandir with alphasort instead of readdir for consistency
During the image generation, When using the e2fsdroid with src_dir,
then dir files are listed with readdir, which will not guarantee
the order across the machines/fielsystems. So instead using the
scandir with alphasort option to list the files in sorted order.
fs->blocksize is int(4 bytes), while data is off_t(8 bytes),
'data_blk = data & ~(fs->blocksize - 1)' will cause data_blk
lose high 4 bytes of data if data > 4G and it'll cause file
inconsistent when using -d option to populate ext4 image file.
[ This was also fixed upstream via 1eec7413677e: "create_inode: fix
copying large files". This commit is just to clean up
whitespace/formatting issues. -- tytso ]
Darrick J. Wong [Thu, 14 Feb 2019 21:02:33 +0000 (16:02 -0500)]
libext2fs: revert "try to always use PUNCH_HOLE for unix_discard"
Revert bcca9876a3428c10417c660b78933e6e70e8a5f5, because
fallocate(PUNCH_HOLE) on block devices was changed to use zeroout
instead of discard shortly after block device fallocate was merged.
zeroout isn't necessarily a "drop storage" operation like discard is,
so we prefer to use that on block devices.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 11 Feb 2019 02:44:05 +0000 (21:44 -0500)]
Check for the newer blkid functions without adding blkid to @LIBS@
If we are using the system-provided blkid, check for the newer blkid
functions without adding -lblkid to @LIBS@. This prevents programs
(like badblocks) which don't use the blkid library getting linked
against it.
David Anderson [Thu, 24 Jan 2019 02:49:23 +0000 (18:49 -0800)]
AOSP: e2fsdroid: Fix incorrect error value handling.
Methods returning 0 or -1 should not pass their return value to com_err.
Also, errno values should not be negated when passed to com_err or
through errcode_t.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bug: N/A
Test: manual test
Ross Burton [Thu, 7 Feb 2019 16:05:13 +0000 (11:05 -0500)]
create_inode: fix copying large files
When copying large files into a ext filesystem at mkfs time the copy
fails at 2^31 bytes in. There are two problems:
copy_file_chunk() passes an offset (off_t, 64-bit typically) to
ext2fs_file_lseek() which expects a ext2_off_t (typedef to __u32) so
the value is truncated. Solve by calling ext2fs_file_llseek() which
takes a u64 offset instead.
try_lseek_copy() rounds the data and hole offsets as found by lseek()
to block boundaries, but the calculation gets truncated to 32-bits.
Solve by casting the 32-bit blocksize to off_t to ensure this doesn't
happen.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
large_dir option allows to create directory with size > 2GB.
debugfs utility outputs negative size value for large directories
and ignores high part of directory size.
Fix debugfs to use high part of directory size in debugfs output
and use appropriate output format.
Theodore Ts'o [Fri, 25 Jan 2019 03:35:15 +0000 (22:35 -0500)]
mke2fs: fix bugs in hugefile creation
For certain sizes mke2fs's hugefile creation would fail with the error:
mke2fs: Could not allocate block in ext2 filesystem while creating huge file 0
This would happen because we had failed to reserve enough space for
the metadata blocks for the hugefile. There were two problems:
1) The overhead calculation function failed to take into account the
cluster size for bigalloc file systems.
2) In the case where num_blocks is 0 and num_files is 1, the overhead
calculation function was passed a size of 0, which caused the
calculated overhead to be zero.
Theodore Ts'o [Fri, 4 Jan 2019 04:06:42 +0000 (23:06 -0500)]
e4defrag: adjust total count when files change during the run
If files are created while e4defrag is running, it's quite possible
for succeed_cnt to be larger than total_count, in which case the
number of failures (calculated via total_count - succeed_cnt) will
overflow and become a very large unsigned number.
The way we calculate statistics is a bit silly, since when recurse
into directories is counted, it's counted as a "failure". But we'll
ignore this from now, and avoid the unsigned overflow.
This wasn't really the right fix, since there can't be more than 2**32
files in a file system. The real issue is when the number of files in
a directory change during the e4defrag run.
Theodore Ts'o [Thu, 3 Jan 2019 19:10:32 +0000 (14:10 -0500)]
tests: generate test_data.tmp file from a script
Even if the debugfs executable is not sparse, it might contain a 1k
aligned block which is all zeros, which when written into a 1k block
file system by a test like d_loaddump will result in a sparse file and
causing one or more test failures.
Theodore Ts'o [Sun, 16 Dec 2018 03:13:41 +0000 (22:13 -0500)]
e2fsck: use "Optimize?" prompt for PR_1E_CAN_{COLLAPSE,NARROW}_EXTENT_TREE
If e2fsck can optimize an inode's extent tree, it should ask for
permission to optimize it, not to "fix" it. This was causing some
confusion, since some users interpreted this prompt as an indication
that the file system was inconsistent.
Theodore Ts'o [Thu, 13 Dec 2018 05:53:16 +0000 (00:53 -0500)]
debugfs: fix set_inode_field so it can set the checksum field
Previously, setting the inode field was a no-op, since the library
function ext2fs_write_inode_full() would override whatever value was
set by debugfs. Use the new ext2fs_write_inode2() interface so we can
in fact set the checksum to a potentially wrong value. Also, ignore
the inode checksum failures if we are setting the checksum, and if the
checksum value is "calc", set the inode checksum to the correct value.
e2fsprogs: introduce script used to generate utf8data.h
The mkutf8data program was written by SGI for the kernel. We integrate
it here, but it should be kept in sync with the kernel code, when
applicable. The header generated by this script is also included in
this repository, such that we don't have to carry the UCD files too.
They are only necessary when the header has to be recreated (for
instance, to update the Unicode version supported)
The mkutf8data is not a dependency of utf8data.h because we don't want
the utf8data.h rule to be called after a make clean that would remove
mkutf8data. It is simpler to call it from inside the rule, and make
sure we only trigger the utf8data.h rule if we really need to,
particularly if we decide not to integrate the ucd files in the tree.
The UCD files are not distributed along with this commit, but we add
information on how to fetch them.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 7 Dec 2018 03:44:45 +0000 (22:44 -0500)]
libext2fs: fix regression so we are correctly translating Posix ACL's
The commit 50d0998cfee ("libext2fs: add ea_inode support to set
xattr") broke the fix that was addressed in commit 0ee1eaf70c25
("libext2fs: translate internal ext4 acl to Posix ACL in
ext2fs_xattr_[sg]et()"). This was because although we calculated what
the correct on-disk ACL representation would be, we didn't actually
*store* it, but instead stored the passed-in Posix ACL memory
representation instead.
ext2fs: nls: support UTF-8 11.0 with NFKD normalization
We need this such that we can do normalization and casefolding
compatible with the kernel, in order to properly support fsck
verification and rehashing.
The UTF-8 11.0 implementation is copied and adapted from the kernel code
to ensure maximum compatibility. The decode trie in utf8data.h is
generated using a script and the UCD sources in the kernel code.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
tune2fs: prevent enabling encryption flag on encoding-aware fs
The kernel will refuse to mount filesystems with the encryption and
encoding features enabled at the same time. The encoding feature can
only be set at mke2fs time, so we can just prevent encryption from being
set at a later time by tune2fs.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Basic NLS support is required in e2fsprogs because of fsck, which
needsto calculate dx hashes for encoding aware filesystems. this patch
implements this infrastructure as well as ascii support.
We don't need to do all the dance of versioning as we do in the kernel,
because we know before-hand which encodings and versions we
support (those we know how to store in the sb), so it is simpler just to
create static tables.
This version only has support for ASCII; we'll add support for Unicode
later.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
projects / thirdparty / e2fsprogs.git / log
