Darrick J. Wong [Thu, 14 Feb 2019 21:02:33 +0000 (16:02 -0500)]
libext2fs: revert "try to always use PUNCH_HOLE for unix_discard"
Revert bcca9876a3428c10417c660b78933e6e70e8a5f5, because
fallocate(PUNCH_HOLE) on block devices was changed to use zeroout
instead of discard shortly after block device fallocate was merged.
zeroout isn't necessarily a "drop storage" operation like discard is,
so we prefer to use that on block devices.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 11 Feb 2019 02:44:05 +0000 (21:44 -0500)]
Check for the newer blkid functions without adding blkid to @LIBS@
If we are using the system-provided blkid, check for the newer blkid
functions without adding -lblkid to @LIBS@. This prevents programs
(like badblocks) which don't use the blkid library getting linked
against it.
David Anderson [Thu, 24 Jan 2019 02:49:23 +0000 (18:49 -0800)]
AOSP: e2fsdroid: Fix incorrect error value handling.
Methods returning 0 or -1 should not pass their return value to com_err.
Also, errno values should not be negated when passed to com_err or
through errcode_t.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bug: N/A
Test: manual test
Ross Burton [Thu, 7 Feb 2019 16:05:13 +0000 (11:05 -0500)]
create_inode: fix copying large files
When copying large files into a ext filesystem at mkfs time the copy
fails at 2^31 bytes in. There are two problems:
copy_file_chunk() passes an offset (off_t, 64-bit typically) to
ext2fs_file_lseek() which expects a ext2_off_t (typedef to __u32) so
the value is truncated. Solve by calling ext2fs_file_llseek() which
takes a u64 offset instead.
try_lseek_copy() rounds the data and hole offsets as found by lseek()
to block boundaries, but the calculation gets truncated to 32-bits.
Solve by casting the 32-bit blocksize to off_t to ensure this doesn't
happen.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
large_dir option allows to create directory with size > 2GB.
debugfs utility outputs negative size value for large directories
and ignores high part of directory size.
Fix debugfs to use high part of directory size in debugfs output
and use appropriate output format.
Theodore Ts'o [Fri, 25 Jan 2019 03:35:15 +0000 (22:35 -0500)]
mke2fs: fix bugs in hugefile creation
For certain sizes mke2fs's hugefile creation would fail with the error:
mke2fs: Could not allocate block in ext2 filesystem while creating huge file 0
This would happen because we had failed to reserve enough space for
the metadata blocks for the hugefile. There were two problems:
1) The overhead calculation function failed to take into account the
cluster size for bigalloc file systems.
2) In the case where num_blocks is 0 and num_files is 1, the overhead
calculation function was passed a size of 0, which caused the
calculated overhead to be zero.
Theodore Ts'o [Fri, 4 Jan 2019 04:06:42 +0000 (23:06 -0500)]
e4defrag: adjust total count when files change during the run
If files are created while e4defrag is running, it's quite possible
for succeed_cnt to be larger than total_count, in which case the
number of failures (calculated via total_count - succeed_cnt) will
overflow and become a very large unsigned number.
The way we calculate statistics is a bit silly, since when recurse
into directories is counted, it's counted as a "failure". But we'll
ignore this from now, and avoid the unsigned overflow.
This wasn't really the right fix, since there can't be more than 2**32
files in a file system. The real issue is when the number of files in
a directory change during the e4defrag run.
Theodore Ts'o [Thu, 3 Jan 2019 19:10:32 +0000 (14:10 -0500)]
tests: generate test_data.tmp file from a script
Even if the debugfs executable is not sparse, it might contain a 1k
aligned block which is all zeros, which when written into a 1k block
file system by a test like d_loaddump will result in a sparse file and
causing one or more test failures.
Theodore Ts'o [Sun, 16 Dec 2018 03:13:41 +0000 (22:13 -0500)]
e2fsck: use "Optimize?" prompt for PR_1E_CAN_{COLLAPSE,NARROW}_EXTENT_TREE
If e2fsck can optimize an inode's extent tree, it should ask for
permission to optimize it, not to "fix" it. This was causing some
confusion, since some users interpreted this prompt as an indication
that the file system was inconsistent.
Theodore Ts'o [Thu, 13 Dec 2018 05:53:16 +0000 (00:53 -0500)]
debugfs: fix set_inode_field so it can set the checksum field
Previously, setting the inode field was a no-op, since the library
function ext2fs_write_inode_full() would override whatever value was
set by debugfs. Use the new ext2fs_write_inode2() interface so we can
in fact set the checksum to a potentially wrong value. Also, ignore
the inode checksum failures if we are setting the checksum, and if the
checksum value is "calc", set the inode checksum to the correct value.
e2fsprogs: introduce script used to generate utf8data.h
The mkutf8data program was written by SGI for the kernel. We integrate
it here, but it should be kept in sync with the kernel code, when
applicable. The header generated by this script is also included in
this repository, such that we don't have to carry the UCD files too.
They are only necessary when the header has to be recreated (for
instance, to update the Unicode version supported)
The mkutf8data is not a dependency of utf8data.h because we don't want
the utf8data.h rule to be called after a make clean that would remove
mkutf8data. It is simpler to call it from inside the rule, and make
sure we only trigger the utf8data.h rule if we really need to,
particularly if we decide not to integrate the ucd files in the tree.
The UCD files are not distributed along with this commit, but we add
information on how to fetch them.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 7 Dec 2018 03:44:45 +0000 (22:44 -0500)]
libext2fs: fix regression so we are correctly translating Posix ACL's
The commit 50d0998cfee ("libext2fs: add ea_inode support to set
xattr") broke the fix that was addressed in commit 0ee1eaf70c25
("libext2fs: translate internal ext4 acl to Posix ACL in
ext2fs_xattr_[sg]et()"). This was because although we calculated what
the correct on-disk ACL representation would be, we didn't actually
*store* it, but instead stored the passed-in Posix ACL memory
representation instead.
ext2fs: nls: support UTF-8 11.0 with NFKD normalization
We need this such that we can do normalization and casefolding
compatible with the kernel, in order to properly support fsck
verification and rehashing.
The UTF-8 11.0 implementation is copied and adapted from the kernel code
to ensure maximum compatibility. The decode trie in utf8data.h is
generated using a script and the UCD sources in the kernel code.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
tune2fs: prevent enabling encryption flag on encoding-aware fs
The kernel will refuse to mount filesystems with the encryption and
encoding features enabled at the same time. The encoding feature can
only be set at mke2fs time, so we can just prevent encryption from being
set at a later time by tune2fs.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Basic NLS support is required in e2fsprogs because of fsck, which
needsto calculate dx hashes for encoding aware filesystems. this patch
implements this infrastructure as well as ascii support.
We don't need to do all the dance of versioning as we do in the kernel,
because we know before-hand which encodings and versions we
support (those we know how to store in the sb), so it is simpler just to
create static tables.
This version only has support for ASCII; we'll add support for Unicode
later.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
mke2fs: configure encoding during superblock initialization
This patch implements two new extended options to mkefs, allowing the
user to specify an encoding for file name operations and encoding flags
during filesystem creation. We provide default flags for each encoding,
which the user can overwrite by passing -E fname_encoding-flags to mkfs.
If the user doesn't specify an encoding, the default value from
options.fname_encoding in mke2fs.conf.in file will be used.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libe2p: helpers for configuring the encoding superblock fields
Implement helper functions to convert the encoding name and specific
parameters requested by the user on the command line into the format
that is written to disk.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
...because there could be superblock updates in the journal that haven't
yet been checkpointed. The online scan is perfectly capable of tallying
the free blocks on its own, so teach it do that and make a more accurate
report.
Reported-by: Elana Hashman <Elana.Hashman@twosigma.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 24 Nov 2018 03:34:31 +0000 (22:34 -0500)]
tune2fs: fix false warning that a UUID change will take a long time
If the file system only has the flex_bg feature enabled (with out the
metadata_csum feature enabled), it won't take a long time time fix up
the checksums after changing the UUID. While it does need to
recalculate all of the checksums in the block group descriptors, that
doesn't take a long time.
Also, if the ea_data feature is enabled, changing the UUID will also
take a long time, and we weren't warning the user about that case.
Fix up the warning message so it doesn't mislead people, and is more
accurate.
Theodore Ts'o [Thu, 22 Nov 2018 23:01:56 +0000 (18:01 -0500)]
mk_cmds: don't use explicit pathname for sed
$AWK doesn't use an explicit pathname, and it's perfectly fine to
assume that awk and sed are in the user's PATH. The problem with
using an explicit pathname is that Debian currently allows merged and
non-merged /usr. Avoid using an explicit pathname to prevent
potential problems.
Reserve feature bit and SB field bit for filename encoding
The s_encoding field in the superblock stores a magic number indicating
the encoding format and version used globally by file and directory
names in the filesystem.
The s_encoding_flags defines policies for using the charset encoding,
like how to handle invalid sequences and what kind of normalization to
use.
A feature flag is also allocated to indicate whether this filesystem has
encoding awareness enabled.
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Re-sync the superblock structure declaration with its kernel counterpart
to include the fields added by kernel commit 6a0678a79bb3 ("ext4: super:
extend timestamps to 40 bits")
Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.co.uk> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 5 Nov 2018 17:59:46 +0000 (09:59 -0800)]
e2scrub: fix systemd escaping again
Apparently newer versions of systemd than the one on this author's
laptop <cough> now complain about lack of (path) escaping in unit
instance variable contents:
# e2scrub_all
Scrubbing /home...
Invalid unit name "e2scrub@/home" was escaped as "e2scrub@-home"
(maybe you should use systemd-escape?)
Starting Online ext4 Metadata Check for /home...
So change the escape_path_for_systemd function to escape paths
unconditionally to make the warning go away.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Dongyang [Tue, 13 Nov 2018 04:46:03 +0000 (15:46 +1100)]
e2fsck: check xattr 'system.data' before setting inline_data feature
ext2fs_inline_data_size will happy return 0 and set size to
EXT4_MIN_INLINE_DATA_SIZE even when inode doesn't have
xattr 'system.data', a corrupted i_flags could make e2fsck
enable the inline_data on the superblock.
We should only offer to enable inline_data when i_flags is set
and xattr 'system.data' can be found.
Also use correct prompt for PR_1_INLINE_DATA_FEATURE.
Theodore Ts'o [Sun, 21 Oct 2018 13:32:37 +0000 (09:32 -0400)]
libext2fs: refactor code which fixes up the checksums in an extent tree
The code to recalculate the checksums in an extent tree (which is
needed after an inode is relocated so it has a different inode number)
was duplicated in tune2fs and resize2fs. In addition, this work could
be done in a much more efficient way inside lib/ext2fs/extent.c.
This commit creates a new library function which corrects the
checksums in an inode's extent tree, named: ext2fs_fix_extents_checksums()
Theodore Ts'o [Sun, 21 Oct 2018 11:35:53 +0000 (07:35 -0400)]
e4defrag: handle failure to open the file system gracefully
If e4defrag is run by root, it will try to open the underlying file
system for files that it is trying to defrag so it can get the file
system parameters. It's currently doing this by searching /etc/mtab.
This isn't the best way to go about doing things, but we'll leave it
for now, at least for a maintenance release. (The better way to do
things would be to look up the device using the blkid library, but
that's a more involved change.)
Since the file system parameters isn't strictly speaking necessary
(after all we get by without them when not running as root), we'll
allow e4defrag to continue running if we can't find the file system.
This can happen if /etc/mtab is pointing at /proc/mounts, and the
kernel can't properly identify the root file system, it is reported as
"/dev/root".
Theodore Ts'o [Sat, 20 Oct 2018 13:14:48 +0000 (09:14 -0400)]
resize2fs: update checksums in the extent tree's relocated block
When shrinking an file system, and we need to relocate an inode, the
checksums in its extent tree must get updated to reflect its new inode
number. When doing this, we need to do this *after* we update the
extent tree to reflect any blocks which need to be relocated due to
the file system shrink operation.
Otherwise, in the case where only an interior node of the extent tree
needs to get relocated, and none of the entries in that node need to
be adjusted, the checksum for that interior node is updated in the old
copy of that block, and then after the extent tree is updated to use
the new copy of that interior node, the extent tree is left with an
invalid checksum.
This is a relatively rare case, since it requires the following
conditions to be true:
*) The metadata checksum feature must be enabled.
*) An inode needs to be relocated.
*) The inode needs to have an interior node.
*) The block for that interior node needs to be relocated.
*) None of blocks addressed by entries in that interior node needs
to be relocated.
When all of these conditions are true, though, the file system is left
with corrupted with bad checksum for the extent tree block.
Nicholas Clark [Tue, 16 Oct 2018 19:37:45 +0000 (15:37 -0400)]
fuse2fs: add fakeroot option.
Add a new 'fakeroot' option to fuse2fs. When enabled, fuse2fs will
will pretend to be root when checking file permssions. This allows
fuse2fs to be used for building/modifying rootfs images as an
unprivileged user.
As per the maintainer's request, nosuid and nodev are automatically
enabled when fakeroot is selected (on platforms that support them)
to help prevent accidental misuse.
Signed-off-by: Nicholas Clark <nicholas.clark@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Nicholas Clark [Tue, 16 Oct 2018 19:34:20 +0000 (15:34 -0400)]
fuse2fs: fix fsname option in some cases
FUSE's parser allows command-line options to be specified before
or after the device/image and mount-path. This commit changes the
value of the fsname mount option to be correct even if options are
specified before the target device/image.
Signed-off-by: Nicholas Clark <nicholas.clark@gmail.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Nick Kralevich [Thu, 11 Oct 2018 03:35:17 +0000 (20:35 -0700)]
AOSP: android/perms.c: clean up error handling
There are a number of error conditions which, due to the way
ext2fs_dir_iterate2 operates, would not be propagated to the upper
layers of the call stack. As a result, certain error conditions,
such as not having enough room to allocate blocks for SELinux
labels, would fail silently, instead of causing a compile
failure.
As suggested in
https://android-review.googlesource.com/c/platform/external/e2fsprogs/+/324363
, add a error field to the caller's private data structure, and use the
bit in the field to indicate an error condition. Now, certain errors
which were silently ignored will cause a compile failure when compiling
Android.
Test: Artifically modify selabel_lookup() to return a failure, and
verify Android doesn't compile.
Test: Verify Android compiles under normal circumstances.
Test: Artifically modify ino_add_xattr() to return a failure, and
verify Android doesn't compile.
Bug: 117502873
Bug: 117567573
Bug: 117473440
Lukas Czerner [Tue, 14 Aug 2018 14:37:53 +0000 (16:37 +0200)]
libe2p: avoid segfault when s_nr_users is too high
Currently in e2fsprogs tools it's possible to access out of bounds
memory when reading list of ids sharing a journal log
(journal_superblock_t->s_users[]) in case where s_nr_users is too high.
This is because we never check whether the s_nr_users fits into the
restriction of JFS_USERS_MAX. Fix it by checking that nr_users is not
bigger than JFS_USERS_MAX and error out when possiblem.
Also add test for dumpe2fs. The rest would require involving external
journal which is not possible to test with e2fsprogs test suite at the
moment.
Darrick J. Wong [Tue, 28 Aug 2018 23:38:07 +0000 (16:38 -0700)]
libext2fs: try to always use PUNCH_HOLE for unix_discard
Now that block devices support PUNCH_HOLE via fallocate, refactor the
unix_discard code into a helper that will always try to use it. For
block devices we can fall back to BLKDISCARD, but we prefer to use
fallocate because it will always invalidate the page cache of the zeroed
region.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 28 Aug 2018 23:38:00 +0000 (16:38 -0700)]
libext2fs: use ZERO_RANGE before PUNCH_HOLE in unix_zeroout
In unix_zeroout() for files, we should try a ZERO_RANGE before we try
PUNCH_HOLE because the former will not cause us to lose preallocated
blocks. Since block devices have supported fallocate for a few years
now, refactor the fallocate calls into a helper and call it from either
case.
Reported-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Andreas Dilger <adilger@dilger.ca> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 24 Aug 2018 21:12:20 +0000 (17:12 -0400)]
For --enable-subset, change "make install" so it implies "make install-libs"
The e2fsprogs-libs-1.44.x.tar.gz subset distribution had a hack so
that "make install" would install the libraries via an implied "make
install-libs" --- since after all the tarball had was just the
libraries.
This commit makes "make install" behave the same was as the
e2fsprogs-libs distribution in the case of "configure --enable-subset"
Theodore Ts'o [Wed, 22 Aug 2018 20:33:10 +0000 (16:33 -0400)]
debian: remove hard-coded dependency for libss2 in e2fsprogs.shlibs.local
The hard-coded override is very old, and isn't necessary --- in fact,
it's actively harmful because we are using one or two symbols that
were added to libss2 after e2fsprogs 1.34. So remove it.
Eric Biggers [Tue, 21 Aug 2018 17:59:37 +0000 (10:59 -0700)]
e2fsck: allow verity files to have initialized blocks past i_size
Since ext4 verity is going to be an RO_COMPAT feature rather than an
INCOMPAT one, the on-disk i_size of verity inodes needs to be the data
size rather than the full size. Consequently, verity inodes will have
initialized blocks past i_size, containing the Merkle tree and other
verity metadata. So e2fsck must not fix the i_size of such inodes as it
normally would.
Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Biggers [Tue, 21 Aug 2018 00:15:52 +0000 (17:15 -0700)]
libext2fs: add verity flag to EXT2_LIB_FEATURE_RO_COMPAT_SUPP
The new ro_compat filesystem feature flag for fs-verity was added to
EXT2_FEATURE_RO_COMPAT_SUPP, but that's not actually used by e2fsprogs
itself. So contrary to the v1.44.4 release notes, 'mke2fs -O verity'
doesn't actually work, nor does e2fsck allow the filesystem to have the
verity feature. Fix it by adding the flag to the correct place
(EXT2_LIB_FEATURE_RO_COMPAT_SUPP) too.
Fixes: faae7aa00df0 ("Reserve codepoints for the fsverity feature.") Signed-off-by: Eric Biggers <ebiggers@google.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Upendra [Thu, 1 Mar 2018 08:15:48 +0000 (17:15 +0900)]
AOSP: blkid: Correct the label name for exfat
Volume label name is 16 bit unicode string according to spec.
Currently blkid labels the device without converting it to
utf-8 chars due to which incorrect label is displayed.
projects / thirdparty / e2fsprogs.git / log
