]> git.ipfire.org Git - thirdparty/xfsprogs-dev.git/log
thirdparty/xfsprogs-dev.git
5 years agoxfsprogs: Release v4.16.0 v4.16.0
Eric Sandeen [Thu, 26 Apr 2018 20:13:34 +0000 (15:13 -0500)] 
xfsprogs: Release v4.16.0

Update all the necessary files for a 4.16.0 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: Release v4.16.0-rc1 v4.16.0-rc1
Eric Sandeen [Thu, 12 Apr 2018 21:30:13 +0000 (16:30 -0500)] 
xfsprogs: Release v4.16.0-rc1

Update all the necessary files for a 4.16.0-rc1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: fix getsubopt name definitions to use enums
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_repair: fix getsubopt name definitions to use enums

Convert the getsubopt usage in xfs_repair to use enums and explicitly
initialized array elements, similar to mkfs.  This also fixes the hole
in the o_opts table caused by 42fa89bc1b8dc8 ("xfs_repair: remove
pre_65_beta option") that causes segfaults in xfs/179 and xfs/202.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Fixes: 42fa89bc1b ("xfs_repair: remove pre_65_beta option")
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub_all: use system encoding for lsblk output decoding
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub_all: use system encoding for lsblk output decoding

Don't hardcode utf-8 as the decoding scheme for lsblk output, since the
system could set it to anything else.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub_all: escape paths being passed to systemd service instances
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub_all: escape paths being passed to systemd service instances

systemd doesn't like unit instance names with slashes in them, so it
replaces them with dashes when it invokes the service.  However, it's
not smart enough to convert the dashes to something else, so when it
unescapes the instance name to feed to xfs_scrub, it turns all dashes
into slashes.  "/moo-cow" becomes "-moo-cow" becomes "/moo/cow", which
is wrong.  systemd actually /can/ escape the dashes correctly if it is
told that this is a path (and not a unit name), but it didn't do this
prior to January 2017, so fix this for them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: disable private /tmp for scrub service
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: disable private /tmp for scrub service

Don't make /tmp private when invoking xfs_scrub as a service, because
/tmp might contain or itself be an xfs filesystem mountpoint.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub_all: report version
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub_all: report version

Make xfs_scrub_all -V report its version like the other xfs tools.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: refactor mountpoint finding code to use libfrog path code
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: refactor mountpoint finding code to use libfrog  path code

Use the libfrog path finding code to determine if the argument being
passed in is a mountpoint, remove all mention of taking a block device
(we have never supported that) from the documentation, and fix some
potential memory leaks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: don't warn about confusing names if dir/file only writable by root
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: don't warn about confusing names if dir/file only writable by root

If we are scanning the directory entries or attribute names of a
dir/file and the inode can only be written by root, don't warn about
Unicode confusable names by default because the system administrator
presumably made the system like that.  Also don't warn about really
short confusable names because of the high chance of collisions.  If
the caller really wants all the output, they can run in verbose mode.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: use Unicode skeleton function to find confusing names
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: use Unicode skeleton function to find confusing names

Drop the weak normalization-based Unicode name collision detection in
favor of the confusable name guidelines provided in Unicode TR36 & TR39.
This means that we transform the original name into its Unicode skeleton
in order to do hashing-based collision detection.

The Unicode skeleton is defined as nfd(translation(nfd(string))), which
is to say that it flattens sequences that render ambiguously into a
unambiguous format.  For example, 'l' and '1' can render identically in
some typefaces, so they're both squashed to 'l'.  From the skeletons we
can figure out if two names will look the same, and thereby complain
about them.  The unicode spoofing is provided by libicu, hence the
switch away from libunistring.

Note that potentially confusable names are only worth an informational
warning, since it's entirely possible that with the system typefaces in
use, two names will render distinctly enough that users can tell the
difference.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: check name for suspicious characters
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: check name for suspicious characters

Look for suspicious characters in each name we process.  This includes
control characters, text direction overrides, zero-width code points,
and names that mix characters from different directionalities.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: transition from libunistring to libicu for Unicode processing
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: transition from libunistring to libicu for Unicode processing

Move off of libunistring and onto libicu for Unicode name scanning.
This will make it easy to warn about unicode code points that do not
belong in identifiers (directional overrides, zero width elements) and
warn about names that could render similarly enough to cause confusion.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: make name_entry a first class structure
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: make name_entry a first class structure

Instead of open-coding the construction and hashtable insertion of name
entries, make name_entry a first class object.  This means that we now
have name_entry_ prefix functions that take care of computing Unicode
normalized names as part of name_entry construction, and we pass around
the name_entries when we're looking for suspicious characters and
identically rendering names.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: communicate name problems via flagset instead of booleans
Darrick J. Wong [Thu, 12 Apr 2018 15:34:11 +0000 (10:34 -0500)] 
xfs_scrub: communicate name problems via flagset instead of booleans

Use an unsigned int to pass around name error flags instead of booleans.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: don't complain about different normalization
Darrick J. Wong [Thu, 12 Apr 2018 15:34:08 +0000 (10:34 -0500)] 
xfs_scrub: don't complain about different normalization

Since there are different ways to normalize utf8 names, don't complain
when we find a name that is normalized in a different way than the NFKC
that we use to find duplicate names.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: only run ascii name checks if unicode name checker
Darrick J. Wong [Thu, 12 Apr 2018 15:34:08 +0000 (10:34 -0500)] 
xfs_scrub: only run ascii name checks if unicode name checker

Skip the ASCII name checks if the Unicode name checker is going to run,
since the latter covers everything that the former does.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: avoid buffer overflow when scanning attributes
Darrick J. Wong [Thu, 12 Apr 2018 15:34:08 +0000 (10:34 -0500)] 
xfs_scrub: avoid buffer overflow when scanning attributes

Avoid a buffer overflow when we're formatting extended attribute names
for name checking.  The kernel headers provide us with XATTR_NAME_MAX,
so we can rely on that.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: warn about deprecation of irix, freebsd, darwin
Eric Sandeen [Tue, 3 Apr 2018 16:13:58 +0000 (11:13 -0500)] 
libxfs: warn about deprecation of irix, freebsd, darwin

It's not clear that anyone is using these platforms or if
they even build at this point.  Get someone's attention if
they are trying to use it.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: test XFS_SB_VERSION_SHAREDBIT only once
Eric Sandeen [Tue, 3 Apr 2018 16:13:57 +0000 (11:13 -0500)] 
xfs_repair: test XFS_SB_VERSION_SHAREDBIT only once

Remove 2 of the 3 identical tests for XFS_SB_VERSION_SHAREDBIT

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: remove unused delete_attr_ok
Eric Sandeen [Tue, 3 Apr 2018 16:13:57 +0000 (11:13 -0500)] 
xfsprogs: remove unused delete_attr_ok

delete_attr_ok is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove pre_65_beta option
Eric Sandeen [Tue, 3 Apr 2018 16:13:57 +0000 (11:13 -0500)] 
xfs_repair: remove pre_65_beta option

Irix 6.5 was released 20 years ago.  Remove this option from
the code.  (nb: it's not present in the manpage.)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_shared_allowed variable
Eric Sandeen [Tue, 3 Apr 2018 16:13:57 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_shared_allowed variable

The fs_shared_allowed global was set to 1 and then ignored, and
in fact the feature is never actualy allowed.  Remove it and
the last stragglers of the old features comment.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_has_extflgbit_allowed
Eric Sandeen [Tue, 3 Apr 2018 16:13:57 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_has_extflgbit_allowed

fs_has_extflgbit_allowed is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_sb_feature_bits_allowed
Eric Sandeen [Tue, 3 Apr 2018 16:13:56 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_sb_feature_bits_allowed

fs_sb_feature_bits_allowed is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_aligned_inodes_allowed
Eric Sandeen [Tue, 3 Apr 2018 16:13:56 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_aligned_inodes_allowed

fs_aligned_inodes_allowed is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_has_extflgbit_allowed
Eric Sandeen [Tue, 3 Apr 2018 16:13:56 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_has_extflgbit_allowed

fs_has_extflgbit_allowed is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_attributes2_allowed
Eric Sandeen [Tue, 3 Apr 2018 16:13:56 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_attributes2_allowed

fs_attributes2_allowed is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: remove unused fs_attributes_allowed
Eric Sandeen [Tue, 3 Apr 2018 16:13:49 +0000 (11:13 -0500)] 
xfs_repair: remove unused fs_attributes_allowed

fs_attributes_allowed is never set to anything but 1;
remove it and all associated code.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibfrog: handle NULL dir && blkdev in __fs_table_lookup_mount
Eric Sandeen [Tue, 27 Mar 2018 22:43:37 +0000 (17:43 -0500)] 
libfrog: handle NULL dir && blkdev in __fs_table_lookup_mount

If neither dir nor blkdev is set, dpath never gets set,
and then gets used (uninitalized) later on.

If we are asked where "nothing" is mounted, just return
"nowhere."

Fixes-coverity-id: 1433615
Fixes-coverity-id: 1433616
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: initialize movon in xfs_scrub_connections
Eric Sandeen [Tue, 27 Mar 2018 22:43:37 +0000 (17:43 -0500)] 
xfs_scrub: initialize movon in xfs_scrub_connections

Given the logic in xfs_scrub_connections, it's possible to
fail all 3 tests and wind up checking an uninitialized moveon
variable at the end.  Start out with "true" to avoid this and
move on even if all the conditions in the function are false.

Fixes-coverity-id: 1433617
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: synchronize error levels & logging
Eric Sandeen [Tue, 27 Mar 2018 22:43:37 +0000 (17:43 -0500)] 
xfs_scrub: synchronize error levels & logging

Having only a subset of the five error_levels present in
the log_level[] array is asking for trouble when someone
tries to __str_log(S_PREEN ...) and overruns the array.

Tie it all together in a single structure that's
initialized together to make the mapping more obvious and
idiot-proof.

Fixes-coverity-id: 1433618
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_spaceman: remove incorrect linux/fs.h include
Darrick J. Wong [Tue, 27 Mar 2018 22:43:37 +0000 (17:43 -0500)] 
xfs_spaceman: remove incorrect linux/fs.h include

Remove the direct linux/fs.h include from spaceman because all xfs
utilities should include xfs.h so that we can wrap missing kernel header
declarations properly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agofsck.xfs: allow forced repairs using xfs_repair
Jan Tulak [Tue, 27 Mar 2018 02:27:31 +0000 (21:27 -0500)] 
fsck.xfs: allow forced repairs using xfs_repair

The fsck.xfs script did nothing, because xfs doesn't need a fsck to be
run on every unclean shutdown. However, sometimes it may happen that the
root filesystem really requires the usage of xfs_repair and then it is a
hassle. This patch makes the situation a bit easier by detecting forced
checks (/forcefsck or fsck.mode=force) and invoking xfs_repair.

Signed-off-by: Jan Tulak <jtulak@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: add flag -e to modify exit code for corrected errors
Jan Tulak [Tue, 27 Mar 2018 02:27:31 +0000 (21:27 -0500)] 
xfs_repair: add flag -e to modify exit code for corrected errors

xfs_repair without -n ends with a return code 0 if it finished ok,
no matter if there were some errors in the fs, or not. The new flag
-e means that we can avoid screenscraping and parsing text output to
detect if an error was found (and corrected).

If something could not be corrected or in any other case than the "found
something but fixed it all," the behaviour with this flag is unchanged.

Signed-off-by: Jan Tulak <jtulak@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: make -e and -n exclusivity clear in manpage synopsis]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agometadump/restore: don't use errno after fwrite/fread failures
Eric Sandeen [Tue, 27 Mar 2018 02:27:31 +0000 (21:27 -0500)] 
metadump/restore: don't use errno after fwrite/fread failures

fread/fwrite don't set errno, so printing out strerror(errno)
after a failure leads to incorrect and confusing messages:

# xfs_mdrestore pre_repair.meta pre_repair.img
xfs_mdrestore: error reading from file: Success

Don't return unset errno from write_index, either.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: enable sparse inodes by default
Darrick J. Wong [Tue, 27 Mar 2018 02:27:31 +0000 (21:27 -0500)] 
mkfs: enable sparse inodes by default

Enable the sparse inode feature by default.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_fsr: refactor mountpoint finding to use libfrog paths functions
Darrick J. Wong [Tue, 27 Mar 2018 02:27:31 +0000 (21:27 -0500)] 
xfs_fsr: refactor mountpoint finding to use libfrog paths functions

Refactor the mount-point finding code in fsr to use the libfrog helpers
instead of open-coding yet another routine.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibfrog: fs_table_lookup_mount should realpath the argument
Darrick J. Wong [Tue, 27 Mar 2018 02:27:28 +0000 (21:27 -0500)] 
libfrog: fs_table_lookup_mount should realpath the argument

Call realpath on the dir argument so that we're comparing canonical
paths when looking for the mountpoint.  This fixes the problem where
'/home/' doesn't match '/home' even though they refer to the same thing.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: use custom ifork verifier in mv_orphanage
Eric Sandeen [Tue, 27 Mar 2018 02:27:28 +0000 (21:27 -0500)] 
xfs_repair: use custom ifork verifier in mv_orphanage

Now that we have a custom verifier which can ignore parent
inode numbers, use it in mv_orphanage() as well; orphan inodes
may have invalid parents, and we're about to reconnect
them anyway, so override that test when we get them off disk.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: implement custom ifork verifiers
Darrick J. Wong [Tue, 27 Mar 2018 02:27:28 +0000 (21:27 -0500)] 
xfs_repair: implement custom ifork verifiers

There are a few cases where an early stage of xfs_repair will write an
invalid inode fork buffer to signal to a later stage that it needs to
correct the value.  This happens in phase 4 when we detect an inline
format directory with an invalid .. pointer.  To avoid triggering the
ifork verifiers on this, inject a custom verifier for phase 6 that lets
this pass for now.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: add missing paths header
Baruch Siach [Tue, 27 Mar 2018 02:27:28 +0000 (21:27 -0500)] 
xfs_scrub: add missing paths header

Fix the following build failure with musl libc:

xfs_scrub.c: In function â€˜main’:
xfs_scrub.c:670:11: error: â€˜_PATH_MOUNTED’ undeclared (first use in this function)
    mtab = _PATH_MOUNTED;
           ^~~~~~~~~~~~~

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoworkqueue: add missing pthreads header
Baruch Siach [Tue, 27 Mar 2018 02:27:28 +0000 (21:27 -0500)] 
workqueue: add missing pthreads header

Fix the following build failure with musl libc:

In file included from read_verify.c:25:0:
../include/workqueue.h:39:2: error: unknown type name 'pthread_t'
  pthread_t  *threads;
  ^~~~~~~~~
../include/workqueue.h:42:2: error: unknown type name 'pthread_mutex_t'
  pthread_mutex_t  lock;
  ^~~~~~~~~~~~~~~
../include/workqueue.h:43:2: error: unknown type name 'pthread_cond_t'
  pthread_cond_t  wakeup;
  ^~~~~~~~~~~~~~

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: fix operation time reporting
Dave Chinner [Tue, 27 Mar 2018 02:27:28 +0000 (21:27 -0500)] 
xfs_io: fix operation time reporting

CUrrently the 100th/sec units always report zero, such as:

32 MiB, 8192 ops; 0:00:21.00 (1.476 MiB/sec and 377.9260 ops/sec)
                          ^^

This is incorrect. Fix the maths that is wrong by removing all the
unnecesary floating point maths and just using basic integer
division...

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: don't fail directory repairs when grabbing inodes
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_repair: don't fail directory repairs when grabbing inodes

There are a few places where xfs_repair needs to be able to load a
damaged directory inode to perform repairs.  Since inline data fork
verifiers can now be customized, refactor libxfs_iget to enable
repair to get at this so that we don't crash in phase 6.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_db: print transaction reservation type information
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_db: print transaction reservation type information

Create a new xfs_db command to print the transaction reservation info for
a given filesystem.  This will make it easier to compare the calculations
made by the kernel and xfsprogs in case there is a discrepancy.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: don't try to scan xattrs if bstat says there aren't any
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_scrub: don't try to scan xattrs if bstat says there aren't any

Only try to scan the extended attributes of a file if bstat says that
the file actually has any.  Surprisingly, this reduces the phase 5
runtime by 40% if most of the files don't have attrs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: fix #include ordering to avoid build failure
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_scrub: fix #include ordering to avoid build failure

Fix the ordering of the header includes in all the scrub source.  We put
xfs.h first so that it will pull in include/linux.h which pulls in
linux/fs.h + whatever overrides are necessary (currently limited to
struct fsxattr) to make things work on this platform, and then we remove
the #includes for anything that will get pulled (directly or indirectly)
by xfs.h for cleanliness.  Without this, a user compiling new xfsprogs
on a system with a 4.7 kernel gets this:

Building scrub
    [CC]     disk.o
In file included from ../include/xfs.h:37:0,
                 from disk.c:40:
../include/xfs/linux.h:185:8: error: redefinition of 'struct fsxattr'
 struct fsxattr {
        ^~~~~~~
In file included from disk.c:31:0:
/usr/include/linux/fs.h:155:8: note: originally defined here
 struct fsxattr {
        ^~~~~~~
gmake[2]: *** [../include/buildrules:60: disk.o] Error 1
gmake[1]: *** [include/buildrules:36: scrub] Error 2
make: *** [Makefile:77: default] Error 2

Reported-by: Mikael Magnusson <mikachu@gmail.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: don't ask user to run xfs_repair for only warnings
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_scrub: don't ask user to run xfs_repair for only warnings

Don't advise the user to run xfs_repair on a filesystem that triggers
warnings but no errors; there's no corruption for it to fix.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: log operational messages when interactive
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_scrub: log operational messages when interactive

Record the summary of an interactive session in the system log so that
future support requests can get a better picture of what happened.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_db: don't crash in ablock if there's no inode
Darrick J. Wong [Fri, 9 Mar 2018 02:35:23 +0000 (20:35 -0600)] 
xfs_db: don't crash in ablock if there's no inode

Make sure we actually have an inode selected before trying to unwrap its
attribute fork.  Found via a crash in xfs/288 with project quotas
enabled.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomisc: fix gcc 7.3 warnings
Darrick J. Wong [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
misc: fix gcc 7.3 warnings

Fix various compiler warnings that pop up in 7.3.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: new libxfs-apply option for Signed-off-by: tag
Eric Sandeen [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
xfsprogs: new libxfs-apply option for Signed-off-by: tag

Technically when a maintainer moves a patch from another project,
they should add their Signed-off-by: tag.  Get that info automatically
from git-config, and add an option to to override it if desired,
to make that easy when cross-porting libxfs patches

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: call libxfs_destroy from other utilities
Eric Sandeen [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
xfsprogs: call libxfs_destroy from other utilities

Call libxfs_destroy() from xfs_copy, xfs_db, mkfs.xfs, and
xfs_repair to allow us to detect leaked items in these
utilities as well.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: Catch non-empty zones on destroy
Eric Sandeen [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
libxfs: Catch non-empty zones on destroy

Create and use a kmem_zone_destroy which warns if we are
releasing a non-empty zone when the LIBXFS_LEAK_CHECK
environment variable is set, wire this into libxfs_destroy(),
and call that when various tools exit.

The LIBXFS_LEAK_CHECK environment variable also causes
the program to exit with failure when a leak is detected,
useful for failing automated tests if leaks are encountered.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: move xfs_inode_zone to rdwr.c
Eric Sandeen [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
libxfs: move xfs_inode_zone to rdwr.c

The zone itself is created in rdwr.c, so define it there as
well, and add it to the list of externs in manage_zones along
with all the rest, for consistency.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: add function to free all buffers in bcache
Eric Sandeen [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
libxfs: add function to free all buffers in bcache

libxfs_bcache_purge simply moves all "free" buffers
onto the xfs_buf_freelist mru list; add a new function to
actually free them when we tear everything down, so leak
checkers don't go nuts about lots of unfreed xfs_bufs
at exit.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: Replace XFS_BUF_SET_PTR with xfs_buf_associate_memory
Eric Sandeen [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
libxfs: Replace XFS_BUF_SET_PTR with xfs_buf_associate_memory

We test the return value of the macro, but it returns
returns a side-effect which looks like failure.  Write
a userspace-libxfs-specific version of xfs_buf_associate_memory
to make this code a tad more like the kernel, with a proper
return value to boot.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: add RWF_DSYNC support to pwrite
Dave Chinner [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
xfs_io: add RWF_DSYNC support to pwrite

Enable testing write behaviour with the per-io RWF_DSYNC flag.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: make interpreter explicit to python3
Luis R. Rodriguez [Fri, 9 Mar 2018 02:35:22 +0000 (20:35 -0600)] 
xfs_scrub: make interpreter explicit to python3

Using #!/usr/bin/env makes some package dependency tools
such as rpm complain given that it cannot verify package
dependencies. Making it explicit resolves this lint rant.

Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: Add missing braces to allow zeroing of corrupt log
Xiao Yang [Fri, 9 Mar 2018 02:35:20 +0000 (20:35 -0600)] 
xfs_repair: Add missing braces to allow zeroing of corrupt log

When xlog_find_tail() fails to find the head or the tail, the missing
braces leads that an unparseable log always exits with status 2, even
if we've asked for -n or -L which should proceed.  We can expose this
issue by xfstests case xfs/098.

Fixes: commit b04647edea32 ("xfs_repair: exit with status 2 if log dirtiness is unknown")
Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: support a basic extent swap command
Brian Foster [Fri, 9 Mar 2018 02:35:20 +0000 (20:35 -0600)] 
xfs_io: support a basic extent swap command

Extent swap is a low level mechanism exported by XFS to facilitate
filesystem defragmentation. It is typically invoked by xfs_fsr under
conditions that will atomically adjust inode extent state without
loss of file data.

While xfs_fsr provides some debug capability to tailor its behavior,
it is not flexible enough to facilitate low level tests of the
extent swap mechanism. For example, xfs_fsr may skip swaps between
inodes that consist solely of preallocated extents because it
considers such files already 100% defragmented. Further, xfs_fsr
copies data between files where doing so may be unnecessary and thus
inefficient for lower level tests.

Add a basic swapext command to xfs_io that allows userspace
invocation of the command under more controlled conditions. This
facilites targeted tests without interference from xfs_fsr policy,
such as using files with only preallocated extents, known/expected
failure cases, etc. This command makes no effort to retain data
across the operation. As such, it is for testing purposes only.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: Add missing perror for write_once (-O)
Goldwyn Rodrigues [Fri, 9 Mar 2018 02:35:20 +0000 (20:35 -0600)] 
xfs_io: Add missing perror for write_once (-O)

This got missed in the last set of patches.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomisc: enable link time optimization, if requested
Darrick J. Wong [Fri, 9 Mar 2018 02:35:20 +0000 (20:35 -0600)] 
misc: enable link time optimization, if requested

Enable link time optimization (LTO) if the builder requests it.  The
extra link optimization results in smaller binaries.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomisc: enable retpolines across all xfsprogs utilities
Darrick J. Wong [Fri, 9 Mar 2018 02:35:20 +0000 (20:35 -0600)] 
misc: enable retpolines across all xfsprogs utilities

Detect and enable retpolines for all code, to mitigate Spectre v2
(branch target injection) on x86.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: fix u32 type usage in sb validation function libxfs-4.16-sync
Darrick J. Wong [Wed, 28 Feb 2018 19:46:11 +0000 (13:46 -0600)] 
xfs: fix u32 type usage in sb validation function

Source kernel commit: 131fa58d391fc0939f6c66b23776ad5df5db20f9

Don't use u32, use uint32_t, because this won't work in xfsprogs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: no-op commit, fixed previously to keep build working]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: don't screw up direct writes when freesp is fragmented
Darrick J. Wong [Tue, 27 Feb 2018 04:43:19 +0000 (22:43 -0600)] 
xfs: don't screw up direct writes when freesp is fragmented

Source kernel commit: 6d8a45ce29c7d67cc4fc3016dc2a07660c62482a

xfs_bmap_btalloc is given a range of file offset blocks that must be
allocated to some data/attr/cow fork.  If the fork has an extent size
hint associated with it, the request will be enlarged on both ends to
try to satisfy the alignment hint.  If free space is fragmentated,
sometimes we can allocate some blocks but not enough to fulfill any of
the requested range.  Since bmapi_allocate always trims the new extent
mapping to match the originally requested range, this results in
bmapi_write returning zero and no mapping.

The consequences of this vary -- buffered writes will simply re-call
bmapi_write until it can satisfy at least one block from the original
request.  Direct IO overwrites notice nmaps == 0 and return -ENOSPC
through the dio mechanism out to userspace with the weird result that
writes fail even when we have enough space because the ENOSPC return
overrides any partial write status.  For direct CoW writes the situation
was disastrous because nobody notices us returning an invalid zero-length
wrong-offset mapping to iomap and the write goes off into space.

Therefore, if free space is so fragmented that we managed to allocate
some space but not enough to map into even a single block of the
original allocation request range, we should break the alignment hint in
order to guarantee at least some forward progress for the direct write.
If we return a short allocation to iomap_apply it'll call back about the
remaining blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: treat CoW fork operations as delalloc for quota accounting
Darrick J. Wong [Tue, 27 Feb 2018 04:43:19 +0000 (22:43 -0600)] 
xfs: treat CoW fork operations as delalloc for quota accounting

Source kernel commit: 4b4c1326fd7c7210d23d9dd3bfc51f2b6477bb9e

Since the CoW fork only exists in memory, it is incorrect to update the
on-disk quota block counts when we modify the CoW fork.  Unlike the data
fork, even real extents in the CoW fork are only delalloc-style
reservations (on-disk they're owned by the refcountbt) so they must not
be tracked in the on disk quota info.  Ensure the i_delayed_blks
accounting reflects this too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: refactor accounting updates out of xfs_bmap_btalloc
Darrick J. Wong [Tue, 27 Feb 2018 04:43:19 +0000 (22:43 -0600)] 
xfs: refactor accounting updates out of xfs_bmap_btalloc

Source kernel commit: 751f3767c245f9adf4f0a4f8f04aae9ae1d675a0

Move all the inode and quota accounting updates out of xfs_bmap_btalloc
in preparation for fixing some quota accounting problems with copy on
write.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: refactor inode verifier corruption error printing
Darrick J. Wong [Tue, 27 Feb 2018 04:43:19 +0000 (22:43 -0600)] 
xfs: refactor inode verifier corruption error printing

Source kernel commit: 22431bf3dfbf44d7356933776eb486a6a01dea6f

Refactor inode verifier error reporting into a non-libxfs function so
that we aren't encoding the message format in libxfs.  This also
changes the kernel dmesg output to resemble buffer verifier errors
more closely.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: bmap code cleanup
Shan Hai [Tue, 27 Feb 2018 04:43:19 +0000 (22:43 -0600)] 
xfs: bmap code cleanup

Source kernel commit: 6ca30729c206d62d88730a904af7d543a56273d8

Remove the extent size hint and realtime inode relevant code from
the xfs_bmapi_reserve_delalloc since it is not called on the inode
with extent size hint set or on a realtime inode.

Signed-off-by: Shan Hai <shan.hai@oracle.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoSplit buffer's b_fspriv field
Carlos Maiolino [Tue, 27 Feb 2018 04:43:19 +0000 (22:43 -0600)] 
Split buffer's b_fspriv field

Source kernel commit: fb1755a645972ed096047583600838f6cf414e2b

By splitting the b_fspriv field into two different fields (b_log_item
and b_li_list). It's possible to get rid of an old ABI workaround, by
using the new b_log_item field to store xfs_buf_log_item separated from
the log items attached to the buffer, which will be linked in the new
b_li_list field.

This way, there is no more need to reorder the log items list to place
the buf_log_item at the beginning of the list, simplifying a bit the
logic to handle buffer IO.

This also opens the possibility to change buffer's log items list into a
proper list_head.

b_log_item field is still defined as a void *, because it is still used
by the log buffers to store xlog_in_core structures, and there is no
need to add an extra field on xfs_buf just for xlog_in_core.

Signed-off-by: Carlos Maiolino <cmaiolino@redhat.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
[darrick: minor style changes]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[sandeen: b_li_list unused in userspace]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: convert to new i_version API
Jeff Layton [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: convert to new i_version API

Source kernel commit: f0e28280629e0ec7921f3179409a179b1ea41f24

Signed-off-by: Jeff Layton <jlayton@redhat.com>
Acked-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: check sb_agblocks and sb_agblklog when validating superblock
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: check sb_agblocks and sb_agblklog when validating superblock

Source kernel commit: 4bb73d014785cc55225686f9f46e7192fb59d26b

Currently, we don't check sb_agblocks or sb_agblklog when we validate
the superblock, which means that we can fuzz garbage values into those
values and the mount succeeds.  This leads to all sorts of UBSAN
warnings in xfs/350 since we can then coerce other parts of xfs into
shifting by ridiculously large values.

Once we've validated agblocks, make sure the agcount makes sense.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
[sandeen: fix up u32 usage now so we keep building]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: recheck reflink / dirty page status before freeing CoW reservations
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: recheck reflink / dirty page status before freeing CoW reservations

Source kernel commit: be78ff0e72778eb4df4aac66edb9e97462bfe00d

Eryu Guan reported seeing occasional hangs when running generic/269 with
a new fsstress that supports clonerange/deduperange.  The cause of this
hang is an infinite loop when we convert the CoW fork extents from
unwritten to real just prior to writing the pages out; the infinite
loop happens because there's nothing in the CoW fork to convert, and so
it spins forever.

The fundamental issue here is that when we go to perform these CoW fork
conversions, we're supposed to have an extent waiting for us, but the
low space CoW reaper has snuck in and blown them away!  There are four
conditions that can dissuade the reaper from touching our file -- no
reflink iflag; dirty page cache; writeback in progress; or directio in
progress.  We check the four conditions prior to taking the locks, but
we neglect to recheck them once we have the locks, which is how we end
up whacking the writeback that's in progress.

Therefore, refactor the four checks into a helper function and call it
once again once we have the locks to make sure we really want to reap
the inode.  While we're at it, add an ASSERT for this weird condition so
that we'll fail noisily if we ever screw this up again.

Reported-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Tested-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: btree format ifork loader should check for zero numrecs
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: btree format ifork loader should check for zero numrecs

Source kernel commit: 55e45429ce3e4ac9dd2bf4937b1a499a69ccc4ca

A btree format inode fork with zero records makes no sense, so reject it
if we see it, or else we can miscalculate memory allocations.  Found by
zeroes fuzzing {a,u3}.bmbt.numrecs in xfs/{374,378,412} with KASAN.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: attr leaf verifier needs to check for obviously bad count
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: attr leaf verifier needs to check for obviously bad count

Source kernel commit: 79a69bf8dc240ebeb105226a8a8540df136bf987

In the attribute leaf verifier, we can check for obviously bad values of
firstused and count so that later attempts at lasthash don't run off the
end of the memory buffer.  Found by ones fuzzing hdr.count in xfs/400 with
KASAN.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: directory scrubber must walk through data block to offset
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: directory scrubber must walk through data block to offset

Source kernel commit: ce92d29ddf9908d397895c46b7c78e9db8df414d

In xfs_scrub_dir_rec, we must walk through the directory block entries
to arrive at the offset given by the hash structure.  If we blindly
trust the hash address, we can end up midway into a directory entry and
stray outside the block.  Found by lastbit fuzzing lents[3].address in
xfs/390 with KASAN enabled.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: cross-reference the realtime bitmap
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: cross-reference the realtime bitmap

Source kernel commit: 46d9bfb5e706493777b9dfed666cd8967f69e6fd

While we're scrubbing various btrees, cross-reference the records
with the other metadata.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: add scrub cross-referencing helpers for the refcount btrees
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: add scrub cross-referencing helpers for the refcount btrees

Source kernel commit: 49db55eca5665e32c9d3e67a7d5694bcc6c274de

Add a couple of functions to the refcount btrees that will be used
to cross-reference metadata against the refcountbt.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: add scrub cross-referencing helpers for the rmap btrees
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: add scrub cross-referencing helpers for the rmap btrees

Source kernel commit: ed7c52d4bf92ac1f05b8c251a44a8bf4688f8786

Add a couple of functions to the rmap btrees that will be used
to cross-reference metadata against the rmapbt.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: add scrub cross-referencing helpers for the inode btrees
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: add scrub cross-referencing helpers for the inode btrees

Source kernel commit: 2e001266b67c865ad904e1889658282d0773b207

Add a couple of functions to the inode btrees that will be used
to cross-reference metadata against the inobt.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: add scrub cross-referencing helpers for the free space btrees
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: add scrub cross-referencing helpers for the free space btrees

Source kernel commit: ce1d802e6a889b8ee53b3444c6d7e8cfecadac50

Add a couple of functions to the free space btrees that will be used
to cross-reference metadata against the bnobt/cntbt, and a generic
btree function that provides the real implementation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: cancel tx on xfs_defer_finish() error during xattr set/remove
Brian Foster [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: cancel tx on xfs_defer_finish() error during xattr set/remove

Source kernel commit: c468562879a766de2c2fbedd41b653a7bf4c157d

Chris Dunlop reports a problem where an xattr operation fails,
reports the following error to syslog and hangs during unmount:

================================================
[ BUG: lock held when returning to user space! ]
...
------------------------------------------------
<PID> is leaving the kernel with locks still held!
1 lock held by <PID>:
#0:  (sb_internal){......}, at: [<ffffffffa07692a3>] xfs_trans_alloc+0xe3/0x130 [xfs]

The failure/shutdown occurs during deferred ops processing which
leads to an error return from xfs_defer_finish() via
xfs_attr_leaf_addname(). While the root cause of the failure is
unknown corruption, the cause of the subsequent BUG above and
unmount hang is failure to cancel the transaction before returning
to userspace.

The transaction is not cancelled because the out_defer_cancel error
handling paths in the xfs_attr_[leaf|node]_[add|remove]name()
functions clear args.trans without releasing the transaction. The
callers therefore lose the reference to the transaction and fail to
cancel it.

Since xfs_attr_[set|remove]() always cancel args.trans when != NULL
and xfs_defer_finish()->...->xfs_trans_roll() should always return
with a valid transaction, update the leaf/node xattr functions to
not reset args.trans in the error path responsible for cancelling
deferred ops.

Reported-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: account finobt blocks properly in perag reservation
Brian Foster [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: account finobt blocks properly in perag reservation

Source kernel commit: ad90bb585c45917b6c1bb01c812fba337e689362

XFS started using the perag metadata reservation pool for free inode
btree blocks in commit 76d771b4cbe33 ("xfs: use per-AG reservations
for the finobt"). To handle backwards compatibility, finobt blocks
are accounted against the pool so long as the full reservation is
available at mount time. Otherwise the ->m_inotbt_nores flag is set
and the filesystem falls back to the traditional per-transaction
finobt reservation.

This commit has two problems:

- finobt blocks are always accounted against the metadata
reservation on allocation, regardless of ->m_inotbt_nores state
- finobt blocks are never returned to the reservation pool on free

The first problem affects reflink+finobt filesystems where the full
finobt reservation is not available at mount time. finobt blocks are
essentially stolen from the reflink reservation, putting refcountbt
management at risk of allocation failure. The second problem is an
unconditional leak of metadata reservation whenever finobt is
enabled.

Update the finobt block allocation callouts to consider
->m_inotbt_nores and account blocks appropriately. Blocks should be
consistently accounted against the metadata pool when
->m_inotbt_nores is false and otherwise tagged as RESV_NONE.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: fix check on struct_version for versions 4 or greater
Colin Ian King [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: fix check on struct_version for versions 4 or greater

Source kernel commit: a8789a5ae28f69d7f3791a0e74f8c44222f3108b

It appears that the check for versions 4 or more is incorrect and is
off-by-one. Fix this.

Detected by CoverityScan, CID#1463775 ("Logically dead code")

Fixes: ac503a4cc9e8 ("xfs: refactor the geometry structure filling function")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: use %px for data pointers when debugging
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: use %px for data pointers when debugging

Source kernel commit: c96900435fa9fdfd9702a60cd765bd85e380303e

Starting with commit 57e734423ad ("vsprintf: refactor %pK code out of
pointer"), the behavior of the raw '%p' printk format specifier was
changed to print a 32-bit hash of the pointer value to avoid leaking
kernel pointers into dmesg.  For most situations that's good.

This is /undesirable/ behavior when we're trying to debug XFS, however,
so define a PTR_FMT that prints the actual pointer when we're in debug
mode.

Note that %p for tracepoints still prints the raw pointer, so in the
long run we could consider rewriting some of these messages as
tracepoints.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: change 0x%p -> %p in print messages
Darrick J. Wong [Tue, 27 Feb 2018 04:43:18 +0000 (22:43 -0600)] 
xfs: change 0x%p -> %p in print messages

Source kernel commit: 3d170aa24283568b1ed92a09daa0e05a8788c6a4

Since %p prepends "0x" to the outputted string, we can drop the prefix.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: harden directory integrity checks some more
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: harden directory integrity checks some more

Source kernel commit: 46c59736d8090e602f960aeaf1c6b8292151bf38

If a malicious filesystem image contains a block+ format directory
wherein the directory inode's core.mode is set such that
S_ISDIR(core.mode) == 0, and if there are subdirectories of the
corrupted directory, an attempt to traverse up the directory tree will
crash the kernel in __xfs_dir3_data_check.  Running the online scrub's
parent checks will tend to do this.

The crash occurs because the directory inode's d_ops get set to
xfs_dir[23]_nondir_ops (it's not a directory) but the parent pointer
scrubber's indiscriminate call to xfs_readdir proceeds past the ASSERT
if we have non fatal asserts configured.

Fix the null pointer dereference crash in __xfs_dir3_data_check by
looking for S_ISDIR or wrong d_ops; and teach the parent scrubber
to bail out if it is fed a non-directory "parent".

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: refactor the geometry structure filling function
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: refactor the geometry structure filling function

Source kernel commit: ac503a4cc9e8ab574032e3e217ffb555f5bf2341

Refactor the geometry structure filling function to use the superblock
to fill the fields.  While we're at it, make the function less indenty
and use some whitespace to make the function easier to read.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: hoist xfs_fs_geometry to libxfs
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: hoist xfs_fs_geometry to libxfs

Source kernel commit: c368ebcd4cc3bbc08602adce083ad3cc76a15258

Move xfs_fs_geometry to libxfs so that we can clean up the fs geometry
reporting in xfsprogs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: trace log reservations at mount time
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: trace log reservations at mount time

Source kernel commit: b872af2c8700e9d64af8e13811b7679ede26ca00

At each mount, emit the transaction reservation type information via
tracepoints.  This makes it easier to compare the log reservation info
calculated by the kernel and xfsprogs so that we can more easily diagnose
minimum log size failures on freshly formatted filesystems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: standardize quota verification function outputs
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: standardize quota verification function outputs

Source kernel commit: eebf3cab9c5eac7fdb54fb9e9fb38c06f46f17f3

Rename xfs_dqcheck to xfs_dquot_verify and make it return an
xfs_failaddr_t like every other structure verifier function.
This enables us to check on-disk quotas in the same way that we check
everything else.  Callers are now responsible for logging errors, as
XFS_QMOPT_DOWARN goes away.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: separate dquot repair into a separate function
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: separate dquot repair into a separate function

Source kernel commit: eeea79802871fef82a8ca6ab1220515855e5cdcc

Move the dquot repair code into a separate function and remove
XFS_QMOPT_DQREPAIR in favor of calling the helper directly.  Remove
other dead code because quotacheck is the only caller of DQREPAIR.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: create a new buf_ops pointer to verify structure metadata
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: create a new buf_ops pointer to verify structure metadata

Source kernel commit: b55725974c9d3a5afcdf83daff6fba7d3f91ffca

Expose all metadata structure buffer verifier functions via buf_ops.
These will be used by the online scrub mechanism to look for problems
with buffers that are already sitting around in memory.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: fail out of xfs_attr3_leaf_lookup_int if it looks corrupt
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: fail out of xfs_attr3_leaf_lookup_int if it looks corrupt

Source kernel commit: 8ba92d43d499f4920af983a7c16e02304dd36932

If the xattr leaf block looks corrupt, return -EFSCORRUPTED to userspace
instead of ASSERTing on debug kernels or running off the end of the
buffer on regular kernels.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: provide a centralized method for verifying inline fork data
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: provide a centralized method for verifying inline fork data

Source kernel commit: 9cfb9b47479e237d217dbcfafe034cbf98f45909

Replace the current haphazard dir2 shortform verifier callsites with a
centralized verifier function that can be called either with the default
verifier functions or with a custom set.  This helps us strengthen
integrity checking while providing us with flexibility for repair tools.

xfs_repair wants this to be able to supply its own verifier functions
when trying to fix possibly corrupt metadata.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: refactor short form directory structure verifier function
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: refactor short form directory structure verifier function

Source kernel commit: dc042c2d8ff629dd411e9a60bce9c379e2f8aaf8

Change the short form directory structure verifier function to return
the instruction pointer of a failing check or NULL if everything's ok.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: create structure verifier function for short form symlinks
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: create structure verifier function for short form symlinks

Source kernel commit: 0795e004fd4f2723f3dbf09a195cd7ccf3c74c58

Create a function to check the structure of short form symlink targets.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: create structure verifier function for shortform xattrs
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: create structure verifier function for shortform xattrs

Source kernel commit: 1e1bbd8e7ee0624034e9bf1e91ac11a7aaa2f8a6

Create a function to perform structure verification for short form
extended attributes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: move inode fork verifiers to xfs_dinode_verify
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: move inode fork verifiers to xfs_dinode_verify

Source kernel commit: 71493b839e294065ba63bd6f8d07263f3afee8c6

Consolidate the fork size and format verifiers to xfs_dinode_verify so
that we can reject bad inodes earlier and in a single place.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>