Darrick J. Wong [Mon, 12 Feb 2018 21:47:40 +0000 (15:47 -0600)]
xfs_scrub: remove preen mode
While it's true that the kernel can tell us whether something needs
repairs or it needs optimizing, from the admin's perspective there's
no point in having an optimize-only mode -- either fix everything, or
don't. This is what xfs_repair does w.r.t. -n, so let's do the same
thing too.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Marco Benatto [Mon, 5 Feb 2018 20:43:02 +0000 (14:43 -0600)]
xfs_mdrestore: Don't rewind source file stream
Today, xfs_mdrestore from stdin will fail if the -i flag is
specified, because it attempts to rewind the stream after
the initial read of the metablock. This fails, and
results in an abort with "specified file is not a metadata
dump."
Read the metablock exactly once in main(), validate the magic,
print informational flags if requested, and then pass it to
perform_restore() which will then continue the restore process.
Reported-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Marco A Benatto <marco.antonio.780@gmail.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Marco Benatto [Mon, 5 Feb 2018 20:41:24 +0000 (14:41 -0600)]
xfs_mdrestore: Add -i option to built-in help
Currently we are missing -i option from usage().
This patch adds it to this biult-in help.
Signed-off-by: Marco A Benatto <marco.antonio.780@gmail.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Mon, 5 Feb 2018 20:38:02 +0000 (14:38 -0600)]
xfs_io: fix copy_file_range symbol name collision
glibc 2.27 has a copy_file_range wrapper, so we need to change our
internal function out of the way to avoid compiler warnings.
Reported-by: fredrik@crux.nu Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Mon, 5 Feb 2018 20:37:49 +0000 (14:37 -0600)]
xfs_scrub: kill dead code
We can't reach the return mess at the bottom of __xfs_scrub_test so get
rid of it.
Fixes-coverity-id: 1428798 Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Mon, 5 Feb 2018 20:37:42 +0000 (14:37 -0600)]
xfs_scrub: close dir_fd if we don't get a DIR pointer
If we don't get a directory pointer, close dir_fd before jumping out.
Fixes-coverity-id: 1428799 Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Mon, 5 Feb 2018 20:37:35 +0000 (14:37 -0600)]
mkfs: don't create realtime filesystems with reflink enabled
We don't support reflink on the realtime device, so don't let people
create such things.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)]
xfs_scrub: handle scrub-only kernels more helpfully
If xfs_scrub is run today against a 4.15 kernel, it fails with
EXPERIMENTAL xfs_scrub program in use! Use at your own risk!
Error: /home: Kernel metadata optimization facility is required.
Info: /home: Scrub aborted after phase 1.
/home: 2 errors found.
Be a bit kinder to the user and suggest a path forward. By the
time we fail for missing preen or repair functionality, we do
know that scrub is available, so suggest it.
Further, rather than stating what is required, state what was not
found ... we're failing, so state what was missing, vs. what is
required - seems a bit more definitive.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Bill O'Donnell <billodo@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)]
xfs_scrub: wire up repair ioctl
Create the mechanism we need to actually call the kernel's online repair
functionality. The interface will consume a repair description; the
descriptor management will follow in the next patch.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
libreadline5-dev hasn't existed as a package for quite some time now;
even Debian "oldoldstable" doesn't know what that is. Drop it in favor
of libreadline-gplv2-dev.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: integrate services with systemd
Create a systemd service unit so that we can run the online scrubber
under systemd with (somewhat) appropriate containment.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: create a script to scrub all xfs filesystems
Create an xfs_scrub_all command to find all XFS filesystems
and run an online scrub against them all.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: progress indicator
Implement a progress indicator.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: fstrim the free areas if there are no errors on the filesystem
If the filesystem scan comes out clean or fixes all the problems, call
fstrim to clean out the free areas (if it's an ssd/thinp/whatever).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: check summary counters
Make sure the filesystem summary counters are somewhat close to what
we can find by scanning the filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: optionally use SCSI READ VERIFY commands to scrub data blocks on disk
If we sense that we're talking to a raw SCSI disk, use the SCSI READ
VERIFY command to ask the disk to verify a disk internally. This can
sharply reduce the runtime of the data block verification phase on
devices whose internal bandwidth exceeds their link bandwidth.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: scrub file data blocks
Read all data blocks from the disk, hoping to catch IO errors.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: create infrastructure to read verify data blocks
Manage the scheduling, issuance, and reporting of data block
verification reads. This enables us to combine adjacent (or nearly
adjacent) read requests, and to take advantage of high-IOPS devices by
issuing IO from multiple threads.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: create a bitmap data structure
Create an efficient tree-based bitmap data structure. We will use this
during the data block scan to record the LBAs of IO errors so that we
can report broken files to userspace.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: warn about normalized Unicode name collisions
Iterate all directory and xattr names to look for name collisions
amongst Unicode normalized names. This is generally a sign of buggy
programs or malicious duplicate files.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: warn about suspicious characters in directory/xattr names
Look for control characters and punctuation that interfere with shell
globbing in directory entry names and extended attribute key names.
Technically these aren't filesystem corruptions because names are
arbitrary sequences of bytes, but they've been known to cause problems
in the Unix environment so warn if we see them.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: check directory connectivity
Opening directories by file handle will cause the kernel to perform
parent lookups all the way to the root directory. Take advantage of
this to ensure that directories actually connect to the root. Some
day we'll have parent pointers and can make this more comprehensive.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: scan inodes
Scan all the inodes in the system for problems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: thread-safe stats counter
Create a threaded stats counter that we'll use to track scan progress.
This includes things like how much of the disk blocks we've scanned,
or later how much progress we've made in each phase.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: scan filesystem and AG metadata
Scrub the filesystem and per-AG metadata.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)]
xfs_scrub: wrap the scrub ioctl
Create some wrappers to call the scrub ioctls.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Add a couple of helper functions to estimate the inode and block
counters on the filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: add file space map iteration functions
These helpers enable userspace to iterate all the space map information
for a file. The iteration function uses GETBMAPX.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: add space map iteration functions
These helpers enable userspace to iterate all the space map information
in a filesystem. The iteration function uses GETFSMAP.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: add inode iteration functions
These helpers enable userspace to count or iterate all inodes in a
filesystem. The counting function uses INUMBERS, while the inode
iterator uses INUMBERS and BULKSTAT to iterate over every inode that
should be in the filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: find XFS filesystem geometry
Discover the geometry of the XFS filesystem that we've been told to
scan, and set up some common functions that will be used by the
scrub phases.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: create an abstraction for a block device
Create an abstraction to handle all of our low level disk operations.
We'll eventually use it to bind to a fs mount point and block device.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: figure out how many threads we're going to need
Create the plumbing to figure out how many threads we're going to want
to do all of our scrubbing.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: dispatch the various phases of the scrub program
Create the dispatching routines that we'll use to call out to each
separate phase of the program.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: set up command line argument parsing
Parse command line options in order to set up the context in which we
will scrub the filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: common error handling
Standardize how we record and report errors.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
xfs_scrub: create online filesystem scrub program
Create the foundations of a filesystem scrubbing tool that asks the
kernel to inspect all metadata in the filesystem and (ultimately) to
repair anything that's broken. Also create the man page for the
utility.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)]
mkfs: more sunit/swidth sanity checking
This fixes 2 issues with stripe geometry validation.
# mkfs.xfs -d sunit=64,swidth=0 ...
both data sunit and data swidth options must be specified
But I did specify it, I specified 0!
So use cli_opt_set() to detect that it was specified.
But we can't allow the above configuration (in fact it causes
a % 0 later in mkfs), so catch it in the "swidth must be a
multiple of sunit" test a bit further down.
(sunit=0,swidth=0 /is/ valid, it's used to override disk
geometry if desired.)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)]
mkfs: always explain why numeric inputs are invalid
Always explain why invalid numeric inputs are not valid, and in a
complete sentence since that's what illegal_optio() sets us up for.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Specifying invalid inputs to mkfs does not break any (reasonable) laws,
so the error message should complain about invalid inputs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)]
mkfs: don't crash on dswidth overflow
I ran mkfs.xfs -d su=1048576,sw=$((18 * 1048576)), forgetting that sw
takes a multiple of su (unlike swidth which takes any space unit). I
was surprised when we hit a floating point exception, which I traced
back to an integer overflow when we calculate swidth from dsw.
So, do the 64-bit multiplication so we can detect the overflow and
complain about it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)]
misc: ubsan fixes
Fix a few things the undefined behavior sanitizer complained about.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)]
xfs_db: interpret inode timestamps as signed integers
The Linux kernel treats core.*time.sec as a signed integer value, so
xfs_db should do likewise, or else files will have inconsistent times
if the seconds count is negative.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)]
xfsprogs: update dead urls
Since oss.sgi.com is dead and xfs.org is slowly migrating to
xfs.wiki.kernel.org, update all the documentation links to point to the
current landing pads.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)]
xfs_copy: accept CRC version of ABTB_MAGIC in ASSERT
Not sure how this was missed for so long, but to handle CRC
filesystems, this ASSERT on block magic must accept CRC magic
as well.
Reported-by: Radek Burkat <radek@pinkbike.com> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Bill O'Donnell <billodo@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)]
xfs_logprint: fix v3 inode formatting
Line up flags2/cowextsize line with all the others, using
tabs.
Fixes: 1fe708d60 ("xfs_logprint: support cowextsize reporting in log contents") Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)]
libxfs: use a memory zone for log items
In addition to more closely matching the kernel, this will
help us catch any leaks from these allocations.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)]
libxfs: use a memory zone for transactions
In addition to more closely matching the kernel, this will
help us catch any leaks from these allocations.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)]
libxfs: remove unused buf_fsprivate3
The buf_fsprivate3 field has no actual use, other than a pointless
"if it's not set, set it" in xfs_buf_item_init; nobody cares after
that. Remove it.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Replace the typeless b_fspriv2 and the ugly macros around it with a properly
typed transaction pointer. As a fallout the log buffer state debug checks
are also removed. We could have kept them using casts, but as they do
not have a real purpose we can as well just remove them.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Alex Elder <aelder@sgi.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Alex Elder <aelder@sgi.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Thu, 4 Jan 2018 19:59:53 +0000 (13:59 -0600)]
find-api-violations: fix a broken grep pattern
One of the grep patterns in find-api-violations is mistaken for a
(broken) range specifier when LC_ALL=C, so fix it to work properly.
This was found by wiring up the script to xfstests.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Thu, 4 Jan 2018 19:58:53 +0000 (13:58 -0600)]
xfs_repair: fix libxfs namespace problems
Call libxfs_ functions, not xfs_ functions.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
it errors out with "bad character in field \" and
then ftok_free crashes on an invalid free() because picking up the
previous token (the closing bracket) xrealloc'd the token array to be 5
elements long but never set the last element's tok pointer.
Consequently the ftok_free tries to free whatever garbage pointer is in
that last element and kaboom.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
[sandeen: slightly clarify commit log] Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)]
xfs_db: check should deal with cow staging extents correctly
Teach xfs_check to record cow staging extents correctly. This means that
we strip off the high bit before using startblock.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)]
libhandle: zap fdhash in fshandle_destroy
Set fdhash_head to zero once we've destroyed the handle list to avoid
dangling pointer problems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Ross Zwisler [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)]
xfs_io: add a new 'log_writes' command
Add a new 'log_writes' command to xfs_io so that we can add dm-log-writes
log marks. It's helpful to allow users of xfs_io to adds these marks from
within xfs_io instead of waiting until after xfs_io exits because then they
are able to replay the dm-log-writes log up to immediately after another
xfs_io operation such as mwrite. This isolates the log replay from other
operations that happen as part of xfs_io exiting (file handles being
closed, mmaps being torn down, etc.). This also allows users to insert
multiple marks between different xfs_io commands.
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com> Suggested-by: Dave Chinner <david@fromorbit.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Ross Zwisler [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)]
xfs_io: add MAP_SYNC support to mmap()
Add support for a new -S flag to xfs_io's mmap command. This opens the
mapping with the (MAP_SYNC | MAP_SHARED_VALIDATE) flags instead of the
standard MAP_SHARED flag.
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com> Suggested-by: Dave Chinner <david@fromorbit.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 4 Jan 2018 19:56:50 +0000 (13:56 -0600)]
mkfs: pass switch case value directly into getnum/getstr
Parsing did this sort of thing:
case D_AGCOUNT:
cli->agcount = getnum(value, opts, D_AGCOUNT);
which was just begging for a cut and paste error between the
case value and the enum passed into getnum/getstr. Pass
"subopt" instead so that it is always consistent with the case.
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Thu, 4 Jan 2018 19:56:29 +0000 (13:56 -0600)]
mkfs: un-document removed logarithm based CLI options
Remove logarithm-based options from usage() and manpage.
Fixes: 70f72d5 "mkfs: remove logarithm based CLI options" Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: remove logarithm based CLI options
Very few people use the log2 based size options for various mkfs
parameters and they just clutter up the code. Get rid of them.
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: resolve sector size CLI conflicts
Now we have a two dimensional conflict array, convert the sector
size CLI option conflict determination to use it. To get the error
specification just right, we also need to tweak how we store
and validate the sector size CLI parameter state in the options
table.
Old:
$ mkfs.xfs -N -s size=4k -d sectsize=512 /dev/pmem0
Cannot specify both -d sectsize and -d sectlog
.....
New:
$ mkfs.xfs -N -s size=4k -d sectsize=512 /dev/pmem0
Cannot specify both -s size and -d sectsize
.....
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Eric Sandeen <sandeen@sandeen.net> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: convert subopt name, val pairs to enums and declared arrays
Replace the nasty #define + implicit array index definitions with
pre-declared enums and index specific name array declarations.
This cleans up the code quite a bit and the pre-declaration of the
enums allows tables to use indexes from other tables in things like
conflict specifications.
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: support arbitrary conflict specification
Currently the conflict table is a single dimension, allowing
conflicts to be specified in the same option table. however, we
have conflicts that span option tables (e.g. sector size) and
so we need to encode both the table and the option that conflicts.
Add support for a two dimensional conflict definition and convert
all the code over to use it.
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Eric Sandeen <sandeen@sandeen.net> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: protofile only needs to be set up once
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: simplify minimum log size calculation
mkfs jumps through hoops to call libxfs_log_calc_minimum_size() to
set the minimum log size. We already have a xfs_mount at this point,
we just need to set the superblock up slightly earlier and then mkfs
can call libxfs_log_calc_minimum_size() directly. This means we can
remove mkfs/maxtrres.c completely.
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: use opts parameter during option parsing
Rather than hard coding the global table variable into the
parsing functions.
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: remove use-once default macros
sb_feat was a weird mishmash of hardcoded defaults and macros
(which were used in only this place).
Make it consistent by removing the use-once macros, and remove
the unused XFS_DFL_LOG_SIZE while we're in here.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: document sb_feat_args members
Some of these are more self-explanatory than others ("nci?").
Just mention the bit each one controls, and put them in order
while we're at it. There are more comments about each bit
near its definition.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)]
mkfs: invert project id width boolean name
It's a bit nuts that we have a projid32bit mkfs option, but
we carry around the inverse of its value in "projid16bit" -
just flip it around for sanity.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Wed, 3 Jan 2018 20:10:09 +0000 (14:10 -0600)]
mkfs: remove unused m_uuid in sb_feat_args
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Eric Sandeen [Wed, 3 Jan 2018 20:10:09 +0000 (14:10 -0600)]
mkfs: Don't emit default config message yet
Until we have more than one possible source of configuration,
there is no need to emit the only possibility and clutter
the output. We can decide how it should all look when we
get more than one source.
Apply some i18n to the config description, though.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Calling xfs_rmap_free with an unknown owner is supposed to remove any
rmaps covering that range regardless of owner. This is used by the EFI
recovery code to say "we're freeing this, it mustn't be owned by
anything anymore", but for whatever reason xfs_free_ag_extent filters
them out.
Therefore, remove the filter and make xfs_rmap_unmap actually treat it
as a wildcard owner -- free anything that's already there, and if
there's no owner at all then that's fine too.
There are two existing callers of bmap_add_free that take care the rmap
deferred ops themselves and use OWN_UNKNOWN to skip the EFI-based rmap
cleanup; convert these to use OWN_NULL (via helpers), and now we really
require that an RUI (if any) gets added to the defer ops before any EFI.
Lastly, now that xfs_free_extent filters out OWN_NULL rmap free requests,
growfs will have to consult directly with the rmap to ensure that there
aren't any rmaps in the grown region.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Under the deferred rmap operation scheme, there's a certain order in
which the rmap deferred ops have to be queued to maintain integrity
during log replay. For alloc/map operations that order is cui -> rui;
for free/unmap operations that order is cui -> rui -> efi. However, the
initial refcount code got the ordering wrong in the free side of things
because it queued refcount free op and an EFI and the refcount free op
queued a rmap free op, resulting in the order cui -> efi -> rui.
If we fail before the efd finishes, the efi recovery will try to do a
wildcard rmap removal and the subsequent rui will fail to find the rmap
and blow up. This didn't ever happen due to other screws up in handling
unknown owner rmap removals, but those other screw ups broke recovery in
other ways, so fix the ordering to follow the intended rules.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Move the tracepoint in xfs_iext_insert to after the point where we've
inserted the extent because otherwise we report stale extent data in
the ftrace output.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
In e1a4e37cc7b665 ("xfs: try to avoid blowing out the transaction
reservation when bunmaping a shared extent"), we try to constrain the
amount of real extents we unmap from the data fork in a given call so
that we don't blow out transaction reservations.
However, not all bunmapi operations require a transaction -- if we're
only removing a delalloc extent, no transaction is needed, so we have to
code against that.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
The new attribute leaf buffer is not held locked across the transaction
roll between the shortform->leaf modification and the addition of the
new entry. As a result, the attribute buffer modification being made is
not atomic from an operational perspective. Hence the AIL push can grab
it in the transient state of "just created" after the initial
transaction is rolled, because the buffer has been released. This leads
to xfs_attr3_leaf_verify() asserting that hdr.count is zero, treating
this as in-memory corruption, and shutting down the filesystem.
Darrick ported the original patch to 4.15 and reworked it use the
xfs_defer_bjoin helper and hold/join the buffer correctly across the
second transaction roll.
Signed-off-by: Alex Lyakas <alex@zadarastorage.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
In certain cases, defer_ops callers will lock a buffer and want to hold
the lock across transaction rolls. Similar to ijoined inodes, we want
to dirty & join the buffer with each transaction roll in defer_finish so
that afterwards the caller still owns the buffer lock and we haven't
inadvertently pinned the log.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
xfs_trans_log_buf() is responsible for logging the dirty segments of
a buffer along with setting all of the necessary state on the
transaction, buffer, bli, etc., to ensure that the associated items
are marked as dirty and prepared for I/O. We have a couple use cases
that need to to dirty a buffer in a transaction without actually
logging dirty ranges of the buffer. One existing use case is
ordered buffers, which are currently logged with arbitrary ranges to
accomplish this even though the content of ordered buffers is never
written to the log. Another pending use case is to relog an already
dirty buffer across rolled transactions within the deferred
operations infrastructure. This is required to prevent a held
(XFS_BLI_HOLD) buffer from pinning the tail of the log.
Refactor xfs_trans_log_buf() into a new function that contains all
of the logic responsible to dirty the transaction, lidp, buffer and
bli. This new function can be used in the future for the use cases
outlined above. This patch does not introduce functional changes.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Allison Henderson <allison.henderson@oracle.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
If we create a new file we will need an inode, and usually some metadata
in the parent direction. Aiming for everything to go well despite the
lack of a reservation leads to dirty transactions cancelled under a heavy
create/delete load. This patch removes those nospace transactions, which
will lead to slightly earlier ENOSPC on some workloads, but instead
prevent file system shutdowns due to cancelling dirty transactions for
others.
A customer could observe assertations failures and shutdowns due to
cancelation of dirty transactions during heavy NFS workloads as shown
below:
2017-05-30 21:17:06 kernel: ALERT: [ 2670.728477] XFS (sdb): Internal error xfs_trans_cancel at line 983 of file fs/xfs/xfs_trans.c. Caller xfs_create+0x4ee/0x7d0 [xfs]
2017-05-30 21:17:06 kernel: ALERT: [ 2670.728684] XFS (sdb): Corruption of in-memory data detected. Shutting down filesystem
2017-05-30 21:17:06 kernel: ALERT: [ 2670.728685] XFS (sdb): Please umount the filesystem and rectify the problem(s)
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Use _GOTO instead of _RETURN so we can free the allocated
cursor on error.
Fixes: bf80628 ("xfs: remove xfs_bmse_shift_one")
Fixes-coverity-id: 1423813, 1423676 Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Darrick J. Wong [Thu, 7 Dec 2017 03:24:49 +0000 (21:24 -0600)]
libxfs: libxfs_nproc should never return negative numbers
Prevent libxfs_nproc from returning a negative/zero CPU count if
platform_nproc happens to error out.
Fixes-coverity-id: 1425909
Fixes-coverity-id: 1425910
Fixes-coverity-id: 1425913 Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Dave Chinner [Wed, 6 Dec 2017 23:14:27 +0000 (17:14 -0600)]
mkfs: rework stripe calculations
The data and log stripe calculations a spaghettied all over the mkfs
code. This patch pulls all of the different chunks of code together
into calc_stripe_factors() and removes all the redundant/repeated
checks and calculations that are made.
Signed-Off-By: Dave Chinner <dchinner@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>