]> git.ipfire.org Git - thirdparty/xfsprogs-dev.git/log
thirdparty/xfsprogs-dev.git
6 years agoxfsprogs: Release v4.15.1 v4.15.1
Eric Sandeen [Tue, 27 Feb 2018 01:50:56 +0000 (19:50 -0600)] 
xfsprogs: Release v4.15.1

Update all the necessary files for a 4.15.1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agopo/pl.po: update Polish translation for 4.15.0
Jakub Bogusz [Tue, 27 Feb 2018 01:44:01 +0000 (19:44 -0600)] 
po/pl.po: update Polish translation for 4.15.0

Polish translation update for xfsprogs 4.15.0

Signed-off-by: Jakub Bogusz <qboosh@pld-linux.org>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agodebian: don't fail postinst when upgrading chroot
Darrick J. Wong [Tue, 27 Feb 2018 01:43:58 +0000 (19:43 -0600)] 
debian: don't fail postinst when upgrading chroot

If we're upgrading a systemd-enabled chroot we'll fail because systemctl
can't connect to a running systemd (nor should it).  We don't need to
issue daemon-reload inside a chroot that doesn't have a running systemd,
so we can ignore the return value.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Nathan Scott <nathans@debian.org>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agodebian: add build-depends on pkg-config
Darrick J. Wong [Tue, 27 Feb 2018 01:43:58 +0000 (19:43 -0600)] 
debian: add build-depends on pkg-config

Since the configure scripts now depend on pkg-config to autodetect where
systemd service files go, we need to list pkg-config as a build
dependency.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: Release v4.15.0 v4.15.0
Eric Sandeen [Fri, 23 Feb 2018 19:09:31 +0000 (13:09 -0600)] 
xfsprogs: Release v4.15.0

Update all the necessary files for a 4.15.0 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoFix spelling mistake in mkfs lazy superblock diagnostic
Nathan Scott [Fri, 23 Feb 2018 19:05:20 +0000 (13:05 -0600)] 
Fix spelling mistake in mkfs lazy superblock diagnostic

Signed-off-by: <nathans@debian.org>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoFix grammar and spelling in strings and man pages
Nathan Scott [Fri, 23 Feb 2018 19:05:17 +0000 (13:05 -0600)] 
Fix grammar and spelling in strings and man pages

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoAdd xfs_scrub_fail systemd service file docs entry
Nathan Scott [Fri, 23 Feb 2018 19:05:15 +0000 (13:05 -0600)] 
Add xfs_scrub_fail systemd service file docs entry

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoUpdate .gitignore list for generated scrub cron file
Nathan Scott [Fri, 23 Feb 2018 19:05:13 +0000 (13:05 -0600)] 
Update .gitignore list for generated scrub cron file

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoSeveral updates to use more modern Debian packaging
Nathan Scott [Fri, 23 Feb 2018 19:05:07 +0000 (13:05 -0600)] 
Several updates to use more modern Debian packaging

Switch to Debian packaging features available in more
recent years to resolve some long-standing issues.

In particular, using the quilt format gives non-native
package builds finally, while keeping the ability for
developers to do upstream deb builds.  Also split the
binary-arch and binary-indep debian/rules targets as
is now mandated, and update to latest standard version.
Mark a bunch of long-resolved bugs as fixed in the deb
changelog so they are automatically closed by the next
update.

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: Release v4.15.0-rc1 v4.15.0-rc1
Eric Sandeen [Tue, 13 Feb 2018 19:57:05 +0000 (13:57 -0600)] 
xfsprogs: Release v4.15.0-rc1

Update all the necessary files for a 4.15.0-rc1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: refactor outcome display into a separate helper
Darrick J. Wong [Tue, 13 Feb 2018 19:46:42 +0000 (13:46 -0600)] 
xfs_scrub: refactor outcome display into a separate helper

Move all the printing of the scrub outcome into a separate helper to
declutter the main function.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[sandeen: put "Unmount ..." on its own line]
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: always init phase information
Darrick J. Wong [Mon, 12 Feb 2018 21:47:56 +0000 (15:47 -0600)] 
xfs_scrub: always init phase information

Make sure we initialize the overall phase state before we start
executing any code that can end up in the report-status-and-exit paths.
Otherwise if debugging is turned on we get garbage io/cpu stat reports.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: reclassify some of the warning messages
Darrick J. Wong [Mon, 12 Feb 2018 21:47:54 +0000 (15:47 -0600)] 
xfs_scrub: reclassify some of the warning messages

Some of the warning messages are actually runtime errors in optional
components, so turn them into informational messages.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: reclassify runtime errors
Darrick J. Wong [Mon, 12 Feb 2018 21:47:52 +0000 (15:47 -0600)] 
xfs_scrub: reclassify runtime errors

If the program encounters runtime errors, these should be noted as
information.  Because these errors abort the execution flow (which is
counted as a runtime error), we need only call str_info to log the
event.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: classify lack of ioctl support as a runtime error
Darrick J. Wong [Mon, 12 Feb 2018 21:47:50 +0000 (15:47 -0600)] 
xfs_scrub: classify lack of ioctl support as a runtime error

If the kernel doesn't have the SCRUB_METADATA ioctl that's a runtime
error, not a fs error.  Account it as such.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: remove preen mode
Darrick J. Wong [Mon, 12 Feb 2018 21:47:40 +0000 (15:47 -0600)] 
xfs_scrub: remove preen mode

While it's true that the kernel can tell us whether something needs
repairs or it needs optimizing, from the admin's perspective there's
no point in having an optimize-only mode -- either fix everything, or
don't.  This is what xfs_repair does w.r.t. -n, so let's do the same
thing too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_mdrestore: Don't rewind source file stream
Marco Benatto [Mon, 5 Feb 2018 20:43:02 +0000 (14:43 -0600)] 
xfs_mdrestore: Don't rewind source file stream

Today, xfs_mdrestore from stdin will fail if the -i flag is
specified, because it attempts to rewind the stream after
the initial read of the metablock.  This fails, and
results in an abort with "specified file is not a metadata
dump."

Read the metablock exactly once in main(), validate the magic,
print informational flags if requested, and then pass it to
perform_restore() which will then continue the restore process.

Reported-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Marco A Benatto <marco.antonio.780@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_mdrestore: Add -i option to built-in help
Marco Benatto [Mon, 5 Feb 2018 20:41:24 +0000 (14:41 -0600)] 
xfs_mdrestore: Add -i option to built-in help

Currently we are missing -i option from usage().
This patch adds it to this biult-in help.

Signed-off-by: Marco A Benatto <marco.antonio.780@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: fix copy_file_range symbol name collision
Darrick J. Wong [Mon, 5 Feb 2018 20:38:02 +0000 (14:38 -0600)] 
xfs_io: fix copy_file_range symbol name collision

glibc 2.27 has a copy_file_range wrapper, so we need to change our
internal function out of the way to avoid compiler warnings.

Reported-by: fredrik@crux.nu
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: kill dead code
Darrick J. Wong [Mon, 5 Feb 2018 20:37:49 +0000 (14:37 -0600)] 
xfs_scrub: kill dead code

We can't reach the return mess at the bottom of __xfs_scrub_test so get
rid of it.

Fixes-coverity-id: 1428798
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: close dir_fd if we don't get a DIR pointer
Darrick J. Wong [Mon, 5 Feb 2018 20:37:42 +0000 (14:37 -0600)] 
xfs_scrub: close dir_fd if we don't get a DIR pointer

If we don't get a directory pointer, close dir_fd before jumping out.

Fixes-coverity-id: 1428799
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: don't create realtime filesystems with reflink enabled
Darrick J. Wong [Mon, 5 Feb 2018 20:37:35 +0000 (14:37 -0600)] 
mkfs: don't create realtime filesystems with reflink enabled

We don't support reflink on the realtime device, so don't let people
create such things.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: handle scrub-only kernels more helpfully
Eric Sandeen [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)] 
xfs_scrub: handle scrub-only kernels more helpfully

If xfs_scrub is run today against a 4.15 kernel, it fails with

EXPERIMENTAL xfs_scrub program in use! Use at your own risk!
Error: /home: Kernel metadata optimization facility is required.
Info: /home: Scrub aborted after phase 1.
/home: 2 errors found.

Be a bit kinder to the user and suggest a path forward.  By the
time we fail for missing preen or repair functionality, we do
know that scrub is available, so suggest it.

Further, rather than stating what is required, state what was not
found ... we're failing, so state what was missing, vs. what is
required - seems a bit more definitive.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: wire up repair ioctl
Darrick J. Wong [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)] 
xfs_scrub: wire up repair ioctl

Create the mechanism we need to actually call the kernel's online repair
functionality.  The interface will consume a repair description; the
descriptor management will follow in the next patch.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agodebian/control: remove nonexistent libreadline5-dev build dependency
Darrick J. Wong [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)] 
debian/control: remove nonexistent libreadline5-dev build dependency

libreadline5-dev hasn't existed as a package for quite some time now;
even Debian "oldoldstable" doesn't know what that is.  Drop it in favor
of libreadline-gplv2-dev.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: integrate services with systemd
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: integrate services with systemd

Create a systemd service unit so that we can run the online scrubber
under systemd with (somewhat) appropriate containment.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: create a script to scrub all xfs filesystems
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: create a script to scrub all xfs filesystems

Create an xfs_scrub_all command to find all XFS filesystems
and run an online scrub against them all.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: progress indicator
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: progress indicator

Implement a progress indicator.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: fstrim the free areas if there are no errors on the filesystem
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: fstrim the free areas if there are no errors on the filesystem

If the filesystem scan comes out clean or fixes all the problems, call
fstrim to clean out the free areas (if it's an ssd/thinp/whatever).

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: check summary counters
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: check summary counters

Make sure the filesystem summary counters are somewhat close to what
we can find by scanning the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: optionally use SCSI READ VERIFY commands to scrub data blocks on disk
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: optionally use SCSI READ VERIFY commands to scrub data blocks on disk

If we sense that we're talking to a raw SCSI disk, use the SCSI READ
VERIFY command to ask the disk to verify a disk internally.  This can
sharply reduce the runtime of the data block verification phase on
devices whose internal bandwidth exceeds their link bandwidth.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: scrub file data blocks
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: scrub file data blocks

Read all data blocks from the disk, hoping to catch IO errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: create infrastructure to read verify data blocks
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: create infrastructure to read verify data blocks

Manage the scheduling, issuance, and reporting of data block
verification reads.  This enables us to combine adjacent (or nearly
adjacent) read requests, and to take advantage of high-IOPS devices by
issuing IO from multiple threads.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: create a bitmap data structure
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: create a bitmap data structure

Create an efficient tree-based bitmap data structure.  We will use this
during the data block scan to record the LBAs of IO errors so that we
can report broken files to userspace.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: warn about normalized Unicode name collisions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: warn about normalized Unicode name collisions

Iterate all directory and xattr names to look for name collisions
amongst Unicode normalized names.  This is generally a sign of buggy
programs or malicious duplicate files.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: warn about suspicious characters in directory/xattr names
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: warn about suspicious characters in directory/xattr names

Look for control characters and punctuation that interfere with shell
globbing in directory entry names and extended attribute key names.
Technically these aren't filesystem corruptions because names are
arbitrary sequences of bytes, but they've been known to cause problems
in the Unix environment so warn if we see them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: check directory connectivity
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: check directory connectivity

Opening directories by file handle will cause the kernel to perform
parent lookups all the way to the root directory.  Take advantage of
this to ensure that directories actually connect to the root.  Some
day we'll have parent pointers and can make this more comprehensive.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: scan inodes
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: scan inodes

Scan all the inodes in the system for problems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: thread-safe stats counter
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: thread-safe stats counter

Create a threaded stats counter that we'll use to track scan progress.
This includes things like how much of the disk blocks we've scanned,
or later how much progress we've made in each phase.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: scan filesystem and AG metadata
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: scan filesystem and AG metadata

Scrub the filesystem and per-AG metadata.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: wrap the scrub ioctl
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: wrap the scrub ioctl

Create some wrappers to call the scrub ioctls.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: filesystem counter collection functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: filesystem counter collection functions

Add a couple of helper functions to estimate the inode and block
counters on the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: add file space map iteration functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: add file space map iteration functions

These helpers enable userspace to iterate all the space map information
for a file.  The iteration function uses GETBMAPX.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: add space map iteration functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: add space map iteration functions

These helpers enable userspace to iterate all the space map information
in a filesystem.  The iteration function uses GETFSMAP.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: add inode iteration functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: add inode iteration functions

These helpers enable userspace to count or iterate all inodes in a
filesystem.  The counting function uses INUMBERS, while the inode
iterator uses INUMBERS and BULKSTAT to iterate over every inode that
should be in the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: find XFS filesystem geometry
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: find XFS filesystem geometry

Discover the geometry of the XFS filesystem that we've been told to
scan, and set up some common functions that will be used by the
scrub phases.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: create an abstraction for a block device
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: create an abstraction for a block device

Create an abstraction to handle all of our low level disk operations.
We'll eventually use it to bind to a fs mount point and block device.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: figure out how many threads we're going to need
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: figure out how many threads we're going to need

Create the plumbing to figure out how many threads we're going to want
to do all of our scrubbing.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: dispatch the various phases of the scrub program
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: dispatch the various phases of the scrub program

Create the dispatching routines that we'll use to call out to each
separate phase of the program.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: set up command line argument parsing
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: set up command line argument parsing

Parse command line options in order to set up the context in which we
will scrub the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: common error handling
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: common error handling

Standardize how we record and report errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_scrub: create online filesystem scrub program
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: create online filesystem scrub program

Create the foundations of a filesystem scrubbing tool that asks the
kernel to inspect all metadata in the filesystem and (ultimately) to
repair anything that's broken.  Also create the man page for the
utility.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: more sunit/swidth sanity checking
Eric Sandeen [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
mkfs: more sunit/swidth sanity checking

This fixes 2 issues with stripe geometry validation.

# mkfs.xfs -d sunit=64,swidth=0 ...
both data sunit and data swidth options must be specified

But I did specify it, I specified 0!

So use cli_opt_set() to detect that it was specified.

But we can't allow the above configuration (in fact it causes
a % 0 later in mkfs), so catch it in the "swidth must be a
multiple of sunit" test a bit further down.

(sunit=0,swidth=0 /is/ valid, it's used to override disk
geometry if desired.)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: always explain why numeric inputs are invalid
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
mkfs: always explain why numeric inputs are invalid

Always explain why invalid numeric inputs are not valid, and in a
complete sentence since that's what illegal_optio() sets us up for.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: don't call values 'illegal', they're invalid
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
mkfs: don't call values 'illegal', they're invalid

Specifying invalid inputs to mkfs does not break any (reasonable) laws,
so the error message should complain about invalid inputs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: don't crash on dswidth overflow
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
mkfs: don't crash on dswidth overflow

I ran mkfs.xfs -d su=1048576,sw=$((18 * 1048576)), forgetting that sw
takes a multiple of su (unlike swidth which takes any space unit).  I
was surprised when we hit a floating point exception, which I traced
back to an integer overflow when we calculate swidth from dsw.

So, do the 64-bit multiplication so we can detect the overflow and
complain about it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomisc: ubsan fixes
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
misc: ubsan fixes

Fix a few things the undefined behavior sanitizer complained about.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_db: interpret inode timestamps as signed integers
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
xfs_db: interpret inode timestamps as signed integers

The Linux kernel treats core.*time.sec as a signed integer value, so
xfs_db should do likewise, or else files will have inconsistent times
if the seconds count is negative.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfsprogs: update dead urls
Darrick J. Wong [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
xfsprogs: update dead urls

Since oss.sgi.com is dead and xfs.org is slowly migrating to
xfs.wiki.kernel.org, update all the documentation links to point to the
current landing pads.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_copy: accept CRC version of ABTB_MAGIC in ASSERT
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
xfs_copy: accept CRC version of ABTB_MAGIC in ASSERT

Not sure how this was missed for so long, but to handle CRC
filesystems, this ASSERT on block magic must accept CRC magic
as well.

Reported-by: Radek Burkat <radek@pinkbike.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_logprint: fix v3 inode formatting
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
xfs_logprint: fix v3 inode formatting

Line up flags2/cowextsize line with all the others, using
tabs.

Fixes: 1fe708d60 ("xfs_logprint: support cowextsize reporting in log contents")
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: use a memory zone for log items
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
libxfs: use a memory zone for log items

In addition to more closely matching the kernel, this will
help us catch any leaks from these allocations.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: use a memory zone for transactions
Eric Sandeen [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
libxfs: use a memory zone for transactions

In addition to more closely matching the kernel, this will
help us catch any leaks from these allocations.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibxfs: remove unused buf_fsprivate3
Eric Sandeen [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
libxfs: remove unused buf_fsprivate3

The buf_fsprivate3 field has no actual use, other than a pointless
"if it's not set, set it" in xfs_buf_item_init; nobody cares after
that.  Remove it.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: add a proper transaction pointer to struct xfs_buf
Christoph Hellwig [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
xfs: add a proper transaction pointer to struct xfs_buf

Source kernel commit bf9d9013a2a559858efb590bf922377be9d6d969

Replace the typeless b_fspriv2 and the ugly macros around it with a properly
typed transaction pointer.  As a fallout the log buffer state debug checks
are also removed.  We could have kept them using casts, but as they do
not have a real purpose we can as well just remove them.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Alex Elder <aelder@sgi.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: remove wrappers around b_fspriv
Christoph Hellwig [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
xfs: remove wrappers around b_fspriv

Source kernel commit adadbeefb34f755a3477da51035eeeec2c1fde38

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Alex Elder <aelder@sgi.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agofind-api-violations: fix a broken grep pattern
Darrick J. Wong [Thu, 4 Jan 2018 19:59:53 +0000 (13:59 -0600)] 
find-api-violations: fix a broken grep pattern

One of the grep patterns in find-api-violations is mistaken for a
(broken) range specifier when LC_ALL=C, so fix it to work properly.
This was found by wiring up the script to xfstests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_repair: fix libxfs namespace problems
Darrick J. Wong [Thu, 4 Jan 2018 19:58:53 +0000 (13:58 -0600)] 
xfs_repair: fix libxfs namespace problems

Call libxfs_ functions, not xfs_ functions.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_db: fix crash when field list selector string has trailing slash
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_db: fix crash when field list selector string has trailing slash

If I run the following command:

xfs_db /dev/sdf -x -c 'agf 0' -c 'addr refcntroot' -c 'addr ptrs[1]\'

it errors out with "bad character in field \" and
then ftok_free crashes on an invalid free() because picking up the
previous token (the closing bracket) xrealloc'd the token array to be 5
elements long but never set the last element's tok pointer.
Consequently the ftok_free tries to free whatever garbage pointer is in
that last element and kaboom.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
[sandeen: slightly clarify commit log]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_db: check should deal with cow staging extents correctly
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_db: check should deal with cow staging extents correctly

Teach xfs_check to record cow staging extents correctly.  This means that
we strip off the high bit before using startblock.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agolibhandle: zap fdhash in fshandle_destroy
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
libhandle: zap fdhash in fshandle_destroy

Set fdhash_head to zero once we've destroyed the handle list to avoid
dangling pointer problems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: add a new 'log_writes' command
Ross Zwisler [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_io: add a new 'log_writes' command

Add a new 'log_writes' command to xfs_io so that we can add dm-log-writes
log marks.  It's helpful to allow users of xfs_io to adds these marks from
within xfs_io instead of waiting until after xfs_io exits because then they
are able to replay the dm-log-writes log up to immediately after another
xfs_io operation such as mwrite.  This isolates the log replay from other
operations that happen as part of xfs_io exiting (file handles being
closed, mmaps being torn down, etc.).  This also allows users to insert
multiple marks between different xfs_io commands.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs_io: add MAP_SYNC support to mmap()
Ross Zwisler [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_io: add MAP_SYNC support to mmap()

Add support for a new -S flag to xfs_io's mmap command.  This opens the
mapping with the (MAP_SYNC | MAP_SHARED_VALIDATE) flags instead of the
standard MAP_SHARED flag.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: do not allow both "dev" and "name" subopts for log or realtime
Eric Sandeen [Thu, 4 Jan 2018 19:58:10 +0000 (13:58 -0600)] 
mkfs: do not allow both "dev" and "name" subopts for log or realtime

Todauy this works, with last-parsed-wins semantics:

mkfs.xfs -f -l logdev=/dev/sda1,name=/dev/sda2 /dev/sda3

Disallow it to avoid ambiguity.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: pass switch case value directly into getnum/getstr
Eric Sandeen [Thu, 4 Jan 2018 19:56:50 +0000 (13:56 -0600)] 
mkfs: pass switch case value directly into getnum/getstr

Parsing did this sort of thing:

case D_AGCOUNT:
cli->agcount = getnum(value, opts, D_AGCOUNT);

which was just begging for a cut and paste error between the
case value and the enum passed into getnum/getstr.  Pass
"subopt" instead so that it is always consistent with the case.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: un-document removed logarithm based CLI options
Eric Sandeen [Thu, 4 Jan 2018 19:56:29 +0000 (13:56 -0600)] 
mkfs: un-document removed logarithm based CLI options

Remove logarithm-based options from usage() and manpage.

Fixes: 70f72d5 "mkfs: remove logarithm based CLI options"
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: remove logarithm based CLI options
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: remove logarithm based CLI options

Very few people use the log2 based size options for various mkfs
parameters and they just clutter up the code. Get rid of them.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: resolve sector size CLI conflicts
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: resolve sector size CLI conflicts

Now we have a two dimensional conflict array, convert the sector
size CLI option conflict determination to use it. To get the error
specification just right, we also need to tweak how we store
and validate the sector size CLI parameter state in the options
table.

Old:

$ mkfs.xfs -N -s size=4k -d sectsize=512 /dev/pmem0
Cannot specify both -d sectsize and -d sectlog
.....

New:

$ mkfs.xfs -N -s size=4k -d sectsize=512 /dev/pmem0
Cannot specify both -s size and -d sectsize
.....

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: convert subopt name, val pairs to enums and declared arrays
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: convert subopt name, val pairs to enums and declared arrays

Replace the nasty #define + implicit array index definitions with
pre-declared enums and index specific name array declarations.

This cleans up the code quite a bit and the pre-declaration of the
enums allows tables to use indexes from other tables in things like
conflict specifications.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: support arbitrary conflict specification
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: support arbitrary conflict specification

Currently the conflict table is a single dimension, allowing
conflicts to be specified in the same option table. however, we
have conflicts that span option tables (e.g. sector size) and
so we need to encode both the table and the option that conflicts.

Add support for a two dimensional conflict definition and convert
all the code over to use it.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: protofile only needs to be set up once
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: protofile only needs to be set up once

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: simplify minimum log size calculation
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: simplify minimum log size calculation

mkfs jumps through hoops to call libxfs_log_calc_minimum_size() to
set the minimum log size. We already have a xfs_mount at this point,
we just need to set the superblock up slightly earlier and then mkfs
can call libxfs_log_calc_minimum_size() directly. This means we can
remove mkfs/maxtrres.c completely.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: use opts parameter during option parsing
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: use opts parameter during option parsing

Rather than hard coding the global table variable into the
parsing functions.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: remove use-once default macros
Eric Sandeen [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: remove use-once default macros

sb_feat was a weird mishmash of hardcoded defaults and macros
(which were used in only this place).

Make it consistent by removing the use-once macros, and remove
the unused XFS_DFL_LOG_SIZE while we're in here.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: document sb_feat_args members
Eric Sandeen [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: document sb_feat_args members

Some of these are more self-explanatory than others ("nci?").
Just mention the bit each one controls, and put them in order
while we're at it.  There are more comments about each bit
near its definition.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: invert project id width boolean name
Eric Sandeen [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: invert project id width boolean name

It's a bit nuts that we have a projid32bit mkfs option, but
we carry around the inverse of its value in "projid16bit" -
just flip it around for sanity.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: remove unused m_uuid in sb_feat_args
Eric Sandeen [Wed, 3 Jan 2018 20:10:09 +0000 (14:10 -0600)] 
mkfs: remove unused m_uuid in sb_feat_args

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agomkfs: Don't emit default config message yet
Eric Sandeen [Wed, 3 Jan 2018 20:10:09 +0000 (14:10 -0600)] 
mkfs: Don't emit default config message yet

Until we have more than one possible source of configuration,
there is no need to emit the only possibility and clutter
the output.  We can decide how it should all look when we
get more than one source.

Apply some i18n to the config description, though.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoMerge branch 'libxfs-4.15-sync' into for-next
Eric Sandeen [Wed, 3 Jan 2018 20:09:36 +0000 (14:09 -0600)] 
Merge branch 'libxfs-4.15-sync' into for-next

6 years agoxfs: only skip rmap owner checks for unknown-owner rmap removal libxfs-4.15-sync
Darrick J. Wong [Wed, 3 Jan 2018 19:42:11 +0000 (13:42 -0600)] 
xfs: only skip rmap owner checks for unknown-owner rmap removal

Source kernel commit: 68c58e9b9a88c1a9d0c2eaf6c7acefb00f5fbbfb

For rmap removal, refactor the rmap owner checks into a separate
function, then skip the checks if we are performing an unknown-owner
removal.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: always honor OWN_UNKNOWN rmap removal requests
Darrick J. Wong [Wed, 3 Jan 2018 19:42:11 +0000 (13:42 -0600)] 
xfs: always honor OWN_UNKNOWN rmap removal requests

Source kernel commit: 33df3a9cf925183a6a169bc3eff2bd0febd1298a

Calling xfs_rmap_free with an unknown owner is supposed to remove any
rmaps covering that range regardless of owner.  This is used by the EFI
recovery code to say "we're freeing this, it mustn't be owned by
anything anymore", but for whatever reason xfs_free_ag_extent filters
them out.

Therefore, remove the filter and make xfs_rmap_unmap actually treat it
as a wildcard owner -- free anything that's already there, and if
there's no owner at all then that's fine too.

There are two existing callers of bmap_add_free that take care the rmap
deferred ops themselves and use OWN_UNKNOWN to skip the EFI-based rmap
cleanup; convert these to use OWN_NULL (via helpers), and now we really
require that an RUI (if any) gets added to the defer ops before any EFI.

Lastly, now that xfs_free_extent filters out OWN_NULL rmap free requests,
growfs will have to consult directly with the rmap to ensure that there
aren't any rmaps in the grown region.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: queue deferred rmap ops for cow staging extent alloc/free in the right order
Darrick J. Wong [Wed, 3 Jan 2018 19:42:10 +0000 (13:42 -0600)] 
xfs: queue deferred rmap ops for cow staging extent alloc/free in the right order

Source kernel commit: 0525e952dcceb9fc947c6d395de7f72220c7d081

Under the deferred rmap operation scheme, there's a certain order in
which the rmap deferred ops have to be queued to maintain integrity
during log replay.  For alloc/map operations that order is cui -> rui;
for free/unmap operations that order is cui -> rui -> efi.  However, the
initial refcount code got the ordering wrong in the free side of things
because it queued refcount free op and an EFI and the refcount free op
queued a rmap free op, resulting in the order cui -> efi -> rui.

If we fail before the efd finishes, the efi recovery will try to do a
wildcard rmap removal and the subsequent rui will fail to find the rmap
and blow up.  This didn't ever happen due to other screws up in handling
unknown owner rmap removals, but those other screw ups broke recovery in
other ways, so fix the ordering to follow the intended rules.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: move xfs_iext_insert tracepoint to report useful information
Darrick J. Wong [Wed, 3 Jan 2018 19:42:09 +0000 (13:42 -0600)] 
xfs: move xfs_iext_insert tracepoint to report useful information

Source kernel commit: c54854a437a447a6bb1dcb11f60dd01cef3fa597

Move the tracepoint in xfs_iext_insert to after the point where we've
inserted the extent because otherwise we report stale extent data in
the ftrace output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: account for null transactions in bunmapi
Darrick J. Wong [Wed, 3 Jan 2018 19:42:09 +0000 (13:42 -0600)] 
xfs: account for null transactions in bunmapi

Source kernel commit: 8c57b88637d78a723e0854fc3d06c6d4c31a1e0c

In e1a4e37cc7b665 ("xfs: try to avoid blowing out the transaction
reservation when bunmaping a shared extent"), we try to constrain the
amount of real extents we unmap from the data fork in a given call so
that we don't blow out transaction reservations.

However, not all bunmapi operations require a transaction -- if we're
only removing a delalloc extent, no transaction is needed, so we have to
code against that.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: hold xfs_buf locked between shortform->leaf conversion and the addition of an...
Darrick J. Wong [Wed, 3 Jan 2018 19:42:08 +0000 (13:42 -0600)] 
xfs: hold xfs_buf locked between shortform->leaf conversion and the addition of an attribute

Source kernel commit: 6e643cd094de3bd0f97edcc1db0089afa24d909f

The new attribute leaf buffer is not held locked across the transaction
roll between the shortform->leaf modification and the addition of the
new entry.  As a result, the attribute buffer modification being made is
not atomic from an operational perspective.  Hence the AIL push can grab
it in the transient state of "just created" after the initial
transaction is rolled, because the buffer has been released.  This leads
to xfs_attr3_leaf_verify() asserting that hdr.count is zero, treating
this as in-memory corruption, and shutting down the filesystem.

Darrick ported the original patch to 4.15 and reworked it use the
xfs_defer_bjoin helper and hold/join the buffer correctly across the
second transaction roll.

Signed-off-by: Alex Lyakas <alex@zadarastorage.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: add the ability to join a held buffer to a defer_ops
Darrick J. Wong [Wed, 3 Jan 2018 19:42:03 +0000 (13:42 -0600)] 
xfs: add the ability to join a held buffer to a defer_ops

Source kernel commit: b7b2846fe26f2c0d7f317c874a13d3ecf22670ff

In certain cases, defer_ops callers will lock a buffer and want to hold
the lock across transaction rolls.  Similar to ijoined inodes, we want
to dirty & join the buffer with each transaction roll in defer_finish so
that afterwards the caller still owns the buffer lock and we haven't
inadvertently pinned the log.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: refactor buffer logging into buffer dirtying helper
Brian Foster [Wed, 3 Jan 2018 19:42:03 +0000 (13:42 -0600)] 
xfs: refactor buffer logging into buffer dirtying helper

Source kernel commit: 9684010d38eccda733b61106765e9357cf436f65

xfs_trans_log_buf() is responsible for logging the dirty segments of
a buffer along with setting all of the necessary state on the
transaction, buffer, bli, etc., to ensure that the associated items
are marked as dirty and prepared for I/O. We have a couple use cases
that need to to dirty a buffer in a transaction without actually
logging dirty ranges of the buffer.  One existing use case is
ordered buffers, which are currently logged with arbitrary ranges to
accomplish this even though the content of ordered buffers is never
written to the log. Another pending use case is to relog an already
dirty buffer across rolled transactions within the deferred
operations infrastructure. This is required to prevent a held
(XFS_BLI_HOLD) buffer from pinning the tail of the log.

Refactor xfs_trans_log_buf() into a new function that contains all
of the logic responsible to dirty the transaction, lidp, buffer and
bli. This new function can be used in the future for the use cases
outlined above. This patch does not introduce functional changes.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: remove "no-allocation" reservations for file creations
Christoph Hellwig [Wed, 3 Jan 2018 19:42:03 +0000 (13:42 -0600)] 
xfs: remove "no-allocation" reservations for file creations

Source kernel commit: f59cf5c29919d17b61913c3360a7bd29b72975c1

If we create a new file we will need an inode, and usually some metadata
in the parent direction.  Aiming for everything to go well despite the
lack of a reservation leads to dirty transactions cancelled under a heavy
create/delete load.  This patch removes those nospace transactions, which
will lead to slightly earlier ENOSPC on some workloads, but instead
prevent file system shutdowns due to cancelling dirty transactions for
others.

A customer could observe assertations failures and shutdowns due to
cancelation of dirty transactions during heavy NFS workloads as shown
below:

2017-05-30 21:17:06 kernel: WARNING: [ 2670.728125] XFS: Assertion failed: error != -ENOSPC, file: fs/xfs/xfs_inode.c, line: 1262

2017-05-30 21:17:06 kernel: WARNING: [ 2670.728222] Call Trace:
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728246]  [<ffffffff81795daf>] dump_stack+0x63/0x81
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728262]  [<ffffffff810a1a5a>] warn_slowpath_common+0x8a/0xc0
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728264]  [<ffffffff810a1b8a>] warn_slowpath_null+0x1a/0x20
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728285]  [<ffffffffa01bf403>] asswarn+0x33/0x40 [xfs]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728308]  [<ffffffffa01bb07e>] xfs_create+0x7be/0x7d0 [xfs]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728329]  [<ffffffffa01b6ffb>] xfs_generic_create+0x1fb/0x2e0 [xfs]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728348]  [<ffffffffa01b7114>] xfs_vn_mknod+0x14/0x20 [xfs]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728366]  [<ffffffffa01b7153>] xfs_vn_create+0x13/0x20 [xfs]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728380]  [<ffffffff81231de5>] vfs_create+0xd5/0x140
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728390]  [<ffffffffa045ddb9>] do_nfsd_create+0x499/0x610 [nfsd]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728396]  [<ffffffffa0465fa5>] nfsd3_proc_create+0x135/0x210 [nfsd]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728401]  [<ffffffffa04561e3>] nfsd_dispatch+0xc3/0x210 [nfsd]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728416]  [<ffffffffa03bfa43>] svc_process_common+0x453/0x6f0 [sunrpc]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728423]  [<ffffffffa03bfdf3>] svc_process+0x113/0x1f0 [sunrpc]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728427]  [<ffffffffa0455bcf>] nfsd+0x10f/0x180 [nfsd]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728432]  [<ffffffffa0455ac0>] ? nfsd_destroy+0x80/0x80 [nfsd]
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728438]  [<ffffffff810c0d58>] kthread+0xd8/0xf0
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728441]  [<ffffffff810c0c80>] ? kthread_create_on_node+0x1b0/0x1b0
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728451]  [<ffffffff8179d962>] ret_from_fork+0x42/0x70
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728453]  [<ffffffff810c0c80>] ? kthread_create_on_node+0x1b0/0x1b0
2017-05-30 21:17:06 kernel: WARNING: [ 2670.728454] ---[ end trace f9822c842fec81d4 ]---

2017-05-30 21:17:06 kernel: ALERT: [ 2670.728477] XFS (sdb): Internal error xfs_trans_cancel at line 983 of file fs/xfs/xfs_trans.c.  Caller xfs_create+0x4ee/0x7d0 [xfs]

2017-05-30 21:17:06 kernel: ALERT: [ 2670.728684] XFS (sdb): Corruption of in-memory data detected. Shutting down filesystem
2017-05-30 21:17:06 kernel: ALERT: [ 2670.728685] XFS (sdb): Please umount the filesystem and rectify the problem(s)

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 years agoxfs: fix leaks on corruption errors in xfs_bmap.c
Eric Sandeen [Wed, 3 Jan 2018 19:42:03 +0000 (13:42 -0600)] 
xfs: fix leaks on corruption errors in xfs_bmap.c

Source kernel commit: d41c6172bd4031979eab722c265a2e5764383c3c

Use _GOTO instead of _RETURN so we can free the allocated
cursor on error.

Fixes: bf80628 ("xfs: remove xfs_bmse_shift_one")
Fixes-coverity-id: 14238131423676
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>