]> git.ipfire.org Git - thirdparty/xfsprogs-dev.git/log
thirdparty/xfsprogs-dev.git
7 years agoxfs: refactor short form directory structure verifier function
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: refactor short form directory structure verifier function

Source kernel commit: dc042c2d8ff629dd411e9a60bce9c379e2f8aaf8

Change the short form directory structure verifier function to return
the instruction pointer of a failing check or NULL if everything's ok.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: create structure verifier function for short form symlinks
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: create structure verifier function for short form symlinks

Source kernel commit: 0795e004fd4f2723f3dbf09a195cd7ccf3c74c58

Create a function to check the structure of short form symlink targets.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: create structure verifier function for shortform xattrs
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: create structure verifier function for shortform xattrs

Source kernel commit: 1e1bbd8e7ee0624034e9bf1e91ac11a7aaa2f8a6

Create a function to perform structure verification for short form
extended attributes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: move inode fork verifiers to xfs_dinode_verify
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: move inode fork verifiers to xfs_dinode_verify

Source kernel commit: 71493b839e294065ba63bd6f8d07263f3afee8c6

Consolidate the fork size and format verifiers to xfs_dinode_verify so
that we can reject bad inodes earlier and in a single place.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: verify dinode header first
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: verify dinode header first

Source kernel commit: 50aa90ef03007beca2c9108993f5b4f2bb4f0a66

Move the v3 inode integrity information (crc, owner, metauuid) before we
look at anything else in the inode so that we don't waste time on a torn
write or a totally garbled block.  This makes xfs_dinode_verify more
consistent with the other verifiers.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: refactor verifier callers to print address of failing check
Darrick J. Wong [Tue, 27 Feb 2018 04:43:17 +0000 (22:43 -0600)] 
xfs: refactor verifier callers to print address of failing check

Source kernel commit: bc1a09b8e334bf5fca1d6727aec538dcff957961

Refactor the callers of verifiers to print the instruction address of a
failing check.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: have buffer verifier functions report failing address
Darrick J. Wong [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: have buffer verifier functions report failing address

Source kernel commit: a6a781a58befcbd467ce843af4eaca3906aa1f08

Modify each function that checks the contents of a metadata buffer to
return the instruction address of the failing test so that we can report
more precise failure errors to the log.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: refactor xfs_verifier_error and xfs_buf_ioerror
Darrick J. Wong [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: refactor xfs_verifier_error and xfs_buf_ioerror

Source kernel commit: 31ca03c92c329525ee3a97d99c47f1ebbaed5d63

Since all verification errors also mark the buffer as having an error,
we can combine these two calls.  Later we'll add a xfs_failaddr_t
parameter to promote the idea of reporting corruption errors and the
address of the failing check to enable better debugging reports.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: remove XFS_WANT_CORRUPTED_RETURN from dir3 data verifiers
Darrick J. Wong [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: remove XFS_WANT_CORRUPTED_RETURN from dir3 data verifiers

Source kernel commit: 9101d3707b9acae8bbb0d82d47e99cf5c60b3ee5

Since __xfs_dir3_data_check verifies on-disk metadata, we can't have it
noisily blowing asserts and hanging the system on corrupt data coming in
off the disk.  Instead, have it return a boolean like all the other
checker functions, and only have it noisily fail if we fail in debug
mode.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: refactor short form btree pointer verification
Darrick J. Wong [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: refactor short form btree pointer verification

Source kernel commit: e1e55aaf1cc646b736439cbd5af229759029ae34

Now that we have xfs_verify_agbno, use it to verify short form btree
pointers instead of open-coding them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: refactor long-format btree header verification routines
Darrick J. Wong [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: refactor long-format btree header verification routines

Source kernel commit: 8368a6019d5bbb8b56c140029dcf5ea570b638f1

Create two helper functions to verify the headers of a long format
btree block.  We'll use this later for the realtime rmapbt.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: remove XFS_FSB_SANITY_CHECK
Darrick J. Wong [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: remove XFS_FSB_SANITY_CHECK

Source kernel commit: 59f6fec3bdb2aafc84d39f34000819d232182d71

We already have a function to verify fsb pointers, so get rid of the
last users of the (less robust) macro.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: eliminate duplicate icreate tx reservation functions
Brian Foster [Tue, 27 Feb 2018 04:43:16 +0000 (22:43 -0600)] 
xfs: eliminate duplicate icreate tx reservation functions

Source kernel commit: c017cb5ddfd6326032570d5eba83308c8a9c13a9

The create transaction reservation calculation has two different
branches of code depending on whether the filesystem is a v5 format
fs or older. Each branch considers the max reservation between the
allocation case (new chunk allocation + record insert) and the
modify case (chunk exists, record modification) of inode allocation.

The modify case is the same for both superblock versions with the
exception of the finobt. The finobt helper checks the feature bit,
however, and so the modify case already shares the same code.

Now that inode chunk allocation has been refactored into a helper
that checks the superblock version to calculate the appropriate
reservation for the create transaction, the only remaining
difference between the create and icreate branches is the call to
the finobt helper. As noted above, the finobt helper is a no-op when
the feature is not enabled. Therefore, these branches are
effectively duplicate and can be condensed.

Remove the xfs_calc_create_*() branch of functions and update the
various callers to use the xfs_calc_icreate_*() variant. The latter
creates the same reservation size for v4 create transactions as the
removed branch. As such, this patch does not result in transaction
reservation changes.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: refactor inode chunk alloc/free tx reservation
Brian Foster [Tue, 27 Feb 2018 04:43:15 +0000 (22:43 -0600)] 
xfs: refactor inode chunk alloc/free tx reservation

Source kernel commit: 57af33e451b73f56feb428f5856cdf6e4e0c60cd

The reservation for the various forms of inode allocation is
scattered across several different functions. This includes two
variants of chunk allocation (v5 icreate transactions vs. older
create transactions) and the inode free transaction.

To clean up some of this code and clarify the purpose of specific
allocfree reservations, continue the pattern of defining helper
functions for smaller operational units of broader transactions.
Refactor the reservation into an inode chunk alloc/free helper that
considers the various conditions based on filesystem format.

An inode chunk free involves an extent free and buffer
invalidations. The latter requires reservation for log headers only.
An inode chunk allocation modifies the free space btrees and logs
the chunk on v4 supers. v5 supers initialize the inode chunk using
ordered buffers and so do not log the chunk.

As a side effect of this refactoring, add one more allocfree res to
the ifree transaction. Technically this does not serve a specific
purpose because inode chunks are freed via deferred operations and
thus occur after a transaction roll. tr_ifree has a bit of a history
of tx overruns caused by too many agfl fixups during sustained file
deletion workloads, so add this extra reservation as a form of
padding nonetheless.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: include an allocfree res for inobt modifications
Brian Foster [Tue, 27 Feb 2018 04:43:15 +0000 (22:43 -0600)] 
xfs: include an allocfree res for inobt modifications

Source kernel commit: f03c78f39710995d2766236f229295d91b8de9dd

Analysis of recent reports of log reservation overruns and code
inspection has uncovered that the reservations associated with inode
operations may not cover the worst case scenarios. In particular,
many cases only include one allocfree res. for a particular
operation even though said operations may also entail AGFL fixups
and inode btree block allocations in addition to the actual inode
chunk allocation. This can easily turn into two or three block
allocations (or frees) per operation.

In theory, the only way to define the worst case reservation is to
include an allocfree res for each individual allocation in a
transaction. Since that is impractical (we can perform multiple agfl
fixups per tx and not every allocation results in a full tree
operation), we need to find a reasonable compromise that addresses
the deficiency in practice without blowing out the size of the
transactions.

Since the inode btrees are not filled by the AGFL, record insertion
and removal can directly result in block allocations and frees
depending on the shape of the tree. These allocations and frees
occur in the same transaction context as the inobt update itself,
but are separate from the allocation/free that might be required for
an inode chunk. Therefore, it makes sense to assume that an [f]inobt
insert/remove can directly result in one or more block allocations
on behalf of the tree.

Refactor the inode transaction reservations to include one allocfree
res. per inode btree modification to cover allocations required by
the tree itself. This separates the reservation required to allocate
the inode chunk from the reservation required for inobt record
insertion/removal. Apply the same logic to the finobt. This results
in killing off the finobt modify condition because we no longer
assume that the broader transaction reservation will cover finobt
block allocations and finobt shape changes can occur in either of
the inobt allocation or modify situations.

Suggested-by: Dave Chinner <david@fromorbit.com>
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: truncate transaction does not modify the inobt
Brian Foster [Tue, 27 Feb 2018 04:43:15 +0000 (22:43 -0600)] 
xfs: truncate transaction does not modify the inobt

Source kernel commit: a606ebdb859e78beb757dfefa08001df366e2ef5

The truncate transaction does not ever modify the inode btree, but
includes an associated log reservation. Update
xfs_calc_itruncate_reservation() to remove the reservation
associated with inobt updates.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: fix up agi unlinked list reservations
Brian Foster [Tue, 27 Feb 2018 04:43:15 +0000 (22:43 -0600)] 
xfs: fix up agi unlinked list reservations

Source kernel commit: e8341d9f6348640dff01d8c4a33695dc82bab5a3

The current AGI unlinked list addition and removal reservations do
not reflect the worst case log usage. An unlinked list removal can
log up to two on-disk inode clusters but only includes reservation
for one. An unlinked list addition logs the on-disk cluster but
includes reservation for an in-core inode.

Update the AGI unlinked list reservation helpers to calculate the
correct worst case reservation for the associated operations.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: include inobt buffers in ifree tx log reservation
Brian Foster [Tue, 27 Feb 2018 04:43:15 +0000 (22:43 -0600)] 
xfs: include inobt buffers in ifree tx log reservation

Source kernel commit: a6f485908d5210a5662f7a031bd1deeb3867e466

The tr_ifree transaction handles inode unlinks and inode chunk
frees. The current transaction calculation does not accurately
reflect worst case changes to the inode btree, however. The inobt
portion of the current transaction reservation only covers
modification of a single inobt buffer (for the particular inode
record). This is a historical artifact from the days before XFS
supported full inode chunk removal.

When support for inode chunk removal was added in commit
254f6311ed1b ("Implement deletion of inode clusters in XFS."), the
additional log reservation required for chunk removal was not added
correctly. The new reservation only considered the header overhead
of associated buffers rather than the full contents of the btrees
and AGF and AGFL buffers affected by the transaction. The
reservation for the free space btrees was subsequently fixed up in
ITRUNCATE log reservation"), but the res. for full inobt joins has
never been added.

Further review of the ifree reservation uncovered a couple more
problems:

- The undocumented +2 blocks are intended for the AGF and AGFL, but
are also not sized correctly and should be logged as full sectors
(not FSBs).
- The additional single block header is undocumented and serves no
apparent purpose.

Update xfs_calc_ifree_reservation() to include a full inobt join in
the reservation calculation. Refactor the undocumented blocks
appropriately and fix up the comments to reflect the current
calculation.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfsprogs: Release v4.15.1 v4.15.1
Eric Sandeen [Tue, 27 Feb 2018 01:50:56 +0000 (19:50 -0600)] 
xfsprogs: Release v4.15.1

Update all the necessary files for a 4.15.1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agopo/pl.po: update Polish translation for 4.15.0
Jakub Bogusz [Tue, 27 Feb 2018 01:44:01 +0000 (19:44 -0600)] 
po/pl.po: update Polish translation for 4.15.0

Polish translation update for xfsprogs 4.15.0

Signed-off-by: Jakub Bogusz <qboosh@pld-linux.org>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agodebian: don't fail postinst when upgrading chroot
Darrick J. Wong [Tue, 27 Feb 2018 01:43:58 +0000 (19:43 -0600)] 
debian: don't fail postinst when upgrading chroot

If we're upgrading a systemd-enabled chroot we'll fail because systemctl
can't connect to a running systemd (nor should it).  We don't need to
issue daemon-reload inside a chroot that doesn't have a running systemd,
so we can ignore the return value.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Nathan Scott <nathans@debian.org>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agodebian: add build-depends on pkg-config
Darrick J. Wong [Tue, 27 Feb 2018 01:43:58 +0000 (19:43 -0600)] 
debian: add build-depends on pkg-config

Since the configure scripts now depend on pkg-config to autodetect where
systemd service files go, we need to list pkg-config as a build
dependency.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfsprogs: Release v4.15.0 v4.15.0
Eric Sandeen [Fri, 23 Feb 2018 19:09:31 +0000 (13:09 -0600)] 
xfsprogs: Release v4.15.0

Update all the necessary files for a 4.15.0 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoFix spelling mistake in mkfs lazy superblock diagnostic
Nathan Scott [Fri, 23 Feb 2018 19:05:20 +0000 (13:05 -0600)] 
Fix spelling mistake in mkfs lazy superblock diagnostic

Signed-off-by: <nathans@debian.org>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoFix grammar and spelling in strings and man pages
Nathan Scott [Fri, 23 Feb 2018 19:05:17 +0000 (13:05 -0600)] 
Fix grammar and spelling in strings and man pages

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoAdd xfs_scrub_fail systemd service file docs entry
Nathan Scott [Fri, 23 Feb 2018 19:05:15 +0000 (13:05 -0600)] 
Add xfs_scrub_fail systemd service file docs entry

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoUpdate .gitignore list for generated scrub cron file
Nathan Scott [Fri, 23 Feb 2018 19:05:13 +0000 (13:05 -0600)] 
Update .gitignore list for generated scrub cron file

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoSeveral updates to use more modern Debian packaging
Nathan Scott [Fri, 23 Feb 2018 19:05:07 +0000 (13:05 -0600)] 
Several updates to use more modern Debian packaging

Switch to Debian packaging features available in more
recent years to resolve some long-standing issues.

In particular, using the quilt format gives non-native
package builds finally, while keeping the ability for
developers to do upstream deb builds.  Also split the
binary-arch and binary-indep debian/rules targets as
is now mandated, and update to latest standard version.
Mark a bunch of long-resolved bugs as fixed in the deb
changelog so they are automatically closed by the next
update.

Signed-off-by: Nathan Scott <nathans@debian.org>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfsprogs: Release v4.15.0-rc1 v4.15.0-rc1
Eric Sandeen [Tue, 13 Feb 2018 19:57:05 +0000 (13:57 -0600)] 
xfsprogs: Release v4.15.0-rc1

Update all the necessary files for a 4.15.0-rc1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: refactor outcome display into a separate helper
Darrick J. Wong [Tue, 13 Feb 2018 19:46:42 +0000 (13:46 -0600)] 
xfs_scrub: refactor outcome display into a separate helper

Move all the printing of the scrub outcome into a separate helper to
declutter the main function.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
[sandeen: put "Unmount ..." on its own line]
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: always init phase information
Darrick J. Wong [Mon, 12 Feb 2018 21:47:56 +0000 (15:47 -0600)] 
xfs_scrub: always init phase information

Make sure we initialize the overall phase state before we start
executing any code that can end up in the report-status-and-exit paths.
Otherwise if debugging is turned on we get garbage io/cpu stat reports.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: reclassify some of the warning messages
Darrick J. Wong [Mon, 12 Feb 2018 21:47:54 +0000 (15:47 -0600)] 
xfs_scrub: reclassify some of the warning messages

Some of the warning messages are actually runtime errors in optional
components, so turn them into informational messages.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: reclassify runtime errors
Darrick J. Wong [Mon, 12 Feb 2018 21:47:52 +0000 (15:47 -0600)] 
xfs_scrub: reclassify runtime errors

If the program encounters runtime errors, these should be noted as
information.  Because these errors abort the execution flow (which is
counted as a runtime error), we need only call str_info to log the
event.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: classify lack of ioctl support as a runtime error
Darrick J. Wong [Mon, 12 Feb 2018 21:47:50 +0000 (15:47 -0600)] 
xfs_scrub: classify lack of ioctl support as a runtime error

If the kernel doesn't have the SCRUB_METADATA ioctl that's a runtime
error, not a fs error.  Account it as such.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: remove preen mode
Darrick J. Wong [Mon, 12 Feb 2018 21:47:40 +0000 (15:47 -0600)] 
xfs_scrub: remove preen mode

While it's true that the kernel can tell us whether something needs
repairs or it needs optimizing, from the admin's perspective there's
no point in having an optimize-only mode -- either fix everything, or
don't.  This is what xfs_repair does w.r.t. -n, so let's do the same
thing too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_mdrestore: Don't rewind source file stream
Marco Benatto [Mon, 5 Feb 2018 20:43:02 +0000 (14:43 -0600)] 
xfs_mdrestore: Don't rewind source file stream

Today, xfs_mdrestore from stdin will fail if the -i flag is
specified, because it attempts to rewind the stream after
the initial read of the metablock.  This fails, and
results in an abort with "specified file is not a metadata
dump."

Read the metablock exactly once in main(), validate the magic,
print informational flags if requested, and then pass it to
perform_restore() which will then continue the restore process.

Reported-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Marco A Benatto <marco.antonio.780@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_mdrestore: Add -i option to built-in help
Marco Benatto [Mon, 5 Feb 2018 20:41:24 +0000 (14:41 -0600)] 
xfs_mdrestore: Add -i option to built-in help

Currently we are missing -i option from usage().
This patch adds it to this biult-in help.

Signed-off-by: Marco A Benatto <marco.antonio.780@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_io: fix copy_file_range symbol name collision
Darrick J. Wong [Mon, 5 Feb 2018 20:38:02 +0000 (14:38 -0600)] 
xfs_io: fix copy_file_range symbol name collision

glibc 2.27 has a copy_file_range wrapper, so we need to change our
internal function out of the way to avoid compiler warnings.

Reported-by: fredrik@crux.nu
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: kill dead code
Darrick J. Wong [Mon, 5 Feb 2018 20:37:49 +0000 (14:37 -0600)] 
xfs_scrub: kill dead code

We can't reach the return mess at the bottom of __xfs_scrub_test so get
rid of it.

Fixes-coverity-id: 1428798
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: close dir_fd if we don't get a DIR pointer
Darrick J. Wong [Mon, 5 Feb 2018 20:37:42 +0000 (14:37 -0600)] 
xfs_scrub: close dir_fd if we don't get a DIR pointer

If we don't get a directory pointer, close dir_fd before jumping out.

Fixes-coverity-id: 1428799
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: don't create realtime filesystems with reflink enabled
Darrick J. Wong [Mon, 5 Feb 2018 20:37:35 +0000 (14:37 -0600)] 
mkfs: don't create realtime filesystems with reflink enabled

We don't support reflink on the realtime device, so don't let people
create such things.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: handle scrub-only kernels more helpfully
Eric Sandeen [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)] 
xfs_scrub: handle scrub-only kernels more helpfully

If xfs_scrub is run today against a 4.15 kernel, it fails with

EXPERIMENTAL xfs_scrub program in use! Use at your own risk!
Error: /home: Kernel metadata optimization facility is required.
Info: /home: Scrub aborted after phase 1.
/home: 2 errors found.

Be a bit kinder to the user and suggest a path forward.  By the
time we fail for missing preen or repair functionality, we do
know that scrub is available, so suggest it.

Further, rather than stating what is required, state what was not
found ... we're failing, so state what was missing, vs. what is
required - seems a bit more definitive.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: wire up repair ioctl
Darrick J. Wong [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)] 
xfs_scrub: wire up repair ioctl

Create the mechanism we need to actually call the kernel's online repair
functionality.  The interface will consume a repair description; the
descriptor management will follow in the next patch.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agodebian/control: remove nonexistent libreadline5-dev build dependency
Darrick J. Wong [Fri, 2 Feb 2018 15:32:47 +0000 (09:32 -0600)] 
debian/control: remove nonexistent libreadline5-dev build dependency

libreadline5-dev hasn't existed as a package for quite some time now;
even Debian "oldoldstable" doesn't know what that is.  Drop it in favor
of libreadline-gplv2-dev.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: integrate services with systemd
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: integrate services with systemd

Create a systemd service unit so that we can run the online scrubber
under systemd with (somewhat) appropriate containment.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: create a script to scrub all xfs filesystems
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: create a script to scrub all xfs filesystems

Create an xfs_scrub_all command to find all XFS filesystems
and run an online scrub against them all.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: progress indicator
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: progress indicator

Implement a progress indicator.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: fstrim the free areas if there are no errors on the filesystem
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: fstrim the free areas if there are no errors on the filesystem

If the filesystem scan comes out clean or fixes all the problems, call
fstrim to clean out the free areas (if it's an ssd/thinp/whatever).

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: check summary counters
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: check summary counters

Make sure the filesystem summary counters are somewhat close to what
we can find by scanning the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: optionally use SCSI READ VERIFY commands to scrub data blocks on disk
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: optionally use SCSI READ VERIFY commands to scrub data blocks on disk

If we sense that we're talking to a raw SCSI disk, use the SCSI READ
VERIFY command to ask the disk to verify a disk internally.  This can
sharply reduce the runtime of the data block verification phase on
devices whose internal bandwidth exceeds their link bandwidth.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: scrub file data blocks
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: scrub file data blocks

Read all data blocks from the disk, hoping to catch IO errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: create infrastructure to read verify data blocks
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: create infrastructure to read verify data blocks

Manage the scheduling, issuance, and reporting of data block
verification reads.  This enables us to combine adjacent (or nearly
adjacent) read requests, and to take advantage of high-IOPS devices by
issuing IO from multiple threads.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: create a bitmap data structure
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: create a bitmap data structure

Create an efficient tree-based bitmap data structure.  We will use this
during the data block scan to record the LBAs of IO errors so that we
can report broken files to userspace.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: warn about normalized Unicode name collisions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: warn about normalized Unicode name collisions

Iterate all directory and xattr names to look for name collisions
amongst Unicode normalized names.  This is generally a sign of buggy
programs or malicious duplicate files.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: warn about suspicious characters in directory/xattr names
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: warn about suspicious characters in directory/xattr names

Look for control characters and punctuation that interfere with shell
globbing in directory entry names and extended attribute key names.
Technically these aren't filesystem corruptions because names are
arbitrary sequences of bytes, but they've been known to cause problems
in the Unix environment so warn if we see them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: check directory connectivity
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: check directory connectivity

Opening directories by file handle will cause the kernel to perform
parent lookups all the way to the root directory.  Take advantage of
this to ensure that directories actually connect to the root.  Some
day we'll have parent pointers and can make this more comprehensive.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: scan inodes
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: scan inodes

Scan all the inodes in the system for problems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: thread-safe stats counter
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: thread-safe stats counter

Create a threaded stats counter that we'll use to track scan progress.
This includes things like how much of the disk blocks we've scanned,
or later how much progress we've made in each phase.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: scan filesystem and AG metadata
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: scan filesystem and AG metadata

Scrub the filesystem and per-AG metadata.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: wrap the scrub ioctl
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: wrap the scrub ioctl

Create some wrappers to call the scrub ioctls.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: filesystem counter collection functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:46 +0000 (09:32 -0600)] 
xfs_scrub: filesystem counter collection functions

Add a couple of helper functions to estimate the inode and block
counters on the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: add file space map iteration functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: add file space map iteration functions

These helpers enable userspace to iterate all the space map information
for a file.  The iteration function uses GETBMAPX.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: add space map iteration functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: add space map iteration functions

These helpers enable userspace to iterate all the space map information
in a filesystem.  The iteration function uses GETFSMAP.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: add inode iteration functions
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: add inode iteration functions

These helpers enable userspace to count or iterate all inodes in a
filesystem.  The counting function uses INUMBERS, while the inode
iterator uses INUMBERS and BULKSTAT to iterate over every inode that
should be in the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: find XFS filesystem geometry
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: find XFS filesystem geometry

Discover the geometry of the XFS filesystem that we've been told to
scan, and set up some common functions that will be used by the
scrub phases.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: create an abstraction for a block device
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: create an abstraction for a block device

Create an abstraction to handle all of our low level disk operations.
We'll eventually use it to bind to a fs mount point and block device.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: figure out how many threads we're going to need
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: figure out how many threads we're going to need

Create the plumbing to figure out how many threads we're going to want
to do all of our scrubbing.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: dispatch the various phases of the scrub program
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: dispatch the various phases of the scrub program

Create the dispatching routines that we'll use to call out to each
separate phase of the program.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: set up command line argument parsing
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: set up command line argument parsing

Parse command line options in order to set up the context in which we
will scrub the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: common error handling
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: common error handling

Standardize how we record and report errors.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_scrub: create online filesystem scrub program
Darrick J. Wong [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
xfs_scrub: create online filesystem scrub program

Create the foundations of a filesystem scrubbing tool that asks the
kernel to inspect all metadata in the filesystem and (ultimately) to
repair anything that's broken.  Also create the man page for the
utility.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: more sunit/swidth sanity checking
Eric Sandeen [Fri, 2 Feb 2018 15:32:45 +0000 (09:32 -0600)] 
mkfs: more sunit/swidth sanity checking

This fixes 2 issues with stripe geometry validation.

# mkfs.xfs -d sunit=64,swidth=0 ...
both data sunit and data swidth options must be specified

But I did specify it, I specified 0!

So use cli_opt_set() to detect that it was specified.

But we can't allow the above configuration (in fact it causes
a % 0 later in mkfs), so catch it in the "swidth must be a
multiple of sunit" test a bit further down.

(sunit=0,swidth=0 /is/ valid, it's used to override disk
geometry if desired.)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: always explain why numeric inputs are invalid
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
mkfs: always explain why numeric inputs are invalid

Always explain why invalid numeric inputs are not valid, and in a
complete sentence since that's what illegal_optio() sets us up for.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: don't call values 'illegal', they're invalid
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
mkfs: don't call values 'illegal', they're invalid

Specifying invalid inputs to mkfs does not break any (reasonable) laws,
so the error message should complain about invalid inputs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: don't crash on dswidth overflow
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
mkfs: don't crash on dswidth overflow

I ran mkfs.xfs -d su=1048576,sw=$((18 * 1048576)), forgetting that sw
takes a multiple of su (unlike swidth which takes any space unit).  I
was surprised when we hit a floating point exception, which I traced
back to an integer overflow when we calculate swidth from dsw.

So, do the 64-bit multiplication so we can detect the overflow and
complain about it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomisc: ubsan fixes
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
misc: ubsan fixes

Fix a few things the undefined behavior sanitizer complained about.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_db: interpret inode timestamps as signed integers
Darrick J. Wong [Fri, 2 Feb 2018 15:32:43 +0000 (09:32 -0600)] 
xfs_db: interpret inode timestamps as signed integers

The Linux kernel treats core.*time.sec as a signed integer value, so
xfs_db should do likewise, or else files will have inconsistent times
if the seconds count is negative.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfsprogs: update dead urls
Darrick J. Wong [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
xfsprogs: update dead urls

Since oss.sgi.com is dead and xfs.org is slowly migrating to
xfs.wiki.kernel.org, update all the documentation links to point to the
current landing pads.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_copy: accept CRC version of ABTB_MAGIC in ASSERT
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
xfs_copy: accept CRC version of ABTB_MAGIC in ASSERT

Not sure how this was missed for so long, but to handle CRC
filesystems, this ASSERT on block magic must accept CRC magic
as well.

Reported-by: Radek Burkat <radek@pinkbike.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_logprint: fix v3 inode formatting
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
xfs_logprint: fix v3 inode formatting

Line up flags2/cowextsize line with all the others, using
tabs.

Fixes: 1fe708d60 ("xfs_logprint: support cowextsize reporting in log contents")
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agolibxfs: use a memory zone for log items
Eric Sandeen [Thu, 25 Jan 2018 19:55:01 +0000 (13:55 -0600)] 
libxfs: use a memory zone for log items

In addition to more closely matching the kernel, this will
help us catch any leaks from these allocations.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agolibxfs: use a memory zone for transactions
Eric Sandeen [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
libxfs: use a memory zone for transactions

In addition to more closely matching the kernel, this will
help us catch any leaks from these allocations.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agolibxfs: remove unused buf_fsprivate3
Eric Sandeen [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
libxfs: remove unused buf_fsprivate3

The buf_fsprivate3 field has no actual use, other than a pointless
"if it's not set, set it" in xfs_buf_item_init; nobody cares after
that.  Remove it.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: add a proper transaction pointer to struct xfs_buf
Christoph Hellwig [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
xfs: add a proper transaction pointer to struct xfs_buf

Source kernel commit bf9d9013a2a559858efb590bf922377be9d6d969

Replace the typeless b_fspriv2 and the ugly macros around it with a properly
typed transaction pointer.  As a fallout the log buffer state debug checks
are also removed.  We could have kept them using casts, but as they do
not have a real purpose we can as well just remove them.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Alex Elder <aelder@sgi.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs: remove wrappers around b_fspriv
Christoph Hellwig [Thu, 25 Jan 2018 19:54:51 +0000 (13:54 -0600)] 
xfs: remove wrappers around b_fspriv

Source kernel commit adadbeefb34f755a3477da51035eeeec2c1fde38

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Alex Elder <aelder@sgi.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agofind-api-violations: fix a broken grep pattern
Darrick J. Wong [Thu, 4 Jan 2018 19:59:53 +0000 (13:59 -0600)] 
find-api-violations: fix a broken grep pattern

One of the grep patterns in find-api-violations is mistaken for a
(broken) range specifier when LC_ALL=C, so fix it to work properly.
This was found by wiring up the script to xfstests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_repair: fix libxfs namespace problems
Darrick J. Wong [Thu, 4 Jan 2018 19:58:53 +0000 (13:58 -0600)] 
xfs_repair: fix libxfs namespace problems

Call libxfs_ functions, not xfs_ functions.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_db: fix crash when field list selector string has trailing slash
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_db: fix crash when field list selector string has trailing slash

If I run the following command:

xfs_db /dev/sdf -x -c 'agf 0' -c 'addr refcntroot' -c 'addr ptrs[1]\'

it errors out with "bad character in field \" and
then ftok_free crashes on an invalid free() because picking up the
previous token (the closing bracket) xrealloc'd the token array to be 5
elements long but never set the last element's tok pointer.
Consequently the ftok_free tries to free whatever garbage pointer is in
that last element and kaboom.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
[sandeen: slightly clarify commit log]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_db: check should deal with cow staging extents correctly
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_db: check should deal with cow staging extents correctly

Teach xfs_check to record cow staging extents correctly.  This means that
we strip off the high bit before using startblock.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agolibhandle: zap fdhash in fshandle_destroy
Darrick J. Wong [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
libhandle: zap fdhash in fshandle_destroy

Set fdhash_head to zero once we've destroyed the handle list to avoid
dangling pointer problems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_io: add a new 'log_writes' command
Ross Zwisler [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_io: add a new 'log_writes' command

Add a new 'log_writes' command to xfs_io so that we can add dm-log-writes
log marks.  It's helpful to allow users of xfs_io to adds these marks from
within xfs_io instead of waiting until after xfs_io exits because then they
are able to replay the dm-log-writes log up to immediately after another
xfs_io operation such as mwrite.  This isolates the log replay from other
operations that happen as part of xfs_io exiting (file handles being
closed, mmaps being torn down, etc.).  This also allows users to insert
multiple marks between different xfs_io commands.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agoxfs_io: add MAP_SYNC support to mmap()
Ross Zwisler [Thu, 4 Jan 2018 19:58:29 +0000 (13:58 -0600)] 
xfs_io: add MAP_SYNC support to mmap()

Add support for a new -S flag to xfs_io's mmap command.  This opens the
mapping with the (MAP_SYNC | MAP_SHARED_VALIDATE) flags instead of the
standard MAP_SHARED flag.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: do not allow both "dev" and "name" subopts for log or realtime
Eric Sandeen [Thu, 4 Jan 2018 19:58:10 +0000 (13:58 -0600)] 
mkfs: do not allow both "dev" and "name" subopts for log or realtime

Todauy this works, with last-parsed-wins semantics:

mkfs.xfs -f -l logdev=/dev/sda1,name=/dev/sda2 /dev/sda3

Disallow it to avoid ambiguity.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: pass switch case value directly into getnum/getstr
Eric Sandeen [Thu, 4 Jan 2018 19:56:50 +0000 (13:56 -0600)] 
mkfs: pass switch case value directly into getnum/getstr

Parsing did this sort of thing:

case D_AGCOUNT:
cli->agcount = getnum(value, opts, D_AGCOUNT);

which was just begging for a cut and paste error between the
case value and the enum passed into getnum/getstr.  Pass
"subopt" instead so that it is always consistent with the case.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: un-document removed logarithm based CLI options
Eric Sandeen [Thu, 4 Jan 2018 19:56:29 +0000 (13:56 -0600)] 
mkfs: un-document removed logarithm based CLI options

Remove logarithm-based options from usage() and manpage.

Fixes: 70f72d5 "mkfs: remove logarithm based CLI options"
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: remove logarithm based CLI options
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: remove logarithm based CLI options

Very few people use the log2 based size options for various mkfs
parameters and they just clutter up the code. Get rid of them.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: resolve sector size CLI conflicts
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: resolve sector size CLI conflicts

Now we have a two dimensional conflict array, convert the sector
size CLI option conflict determination to use it. To get the error
specification just right, we also need to tweak how we store
and validate the sector size CLI parameter state in the options
table.

Old:

$ mkfs.xfs -N -s size=4k -d sectsize=512 /dev/pmem0
Cannot specify both -d sectsize and -d sectlog
.....

New:

$ mkfs.xfs -N -s size=4k -d sectsize=512 /dev/pmem0
Cannot specify both -s size and -d sectsize
.....

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: convert subopt name, val pairs to enums and declared arrays
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: convert subopt name, val pairs to enums and declared arrays

Replace the nasty #define + implicit array index definitions with
pre-declared enums and index specific name array declarations.

This cleans up the code quite a bit and the pre-declaration of the
enums allows tables to use indexes from other tables in things like
conflict specifications.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: support arbitrary conflict specification
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: support arbitrary conflict specification

Currently the conflict table is a single dimension, allowing
conflicts to be specified in the same option table. however, we
have conflicts that span option tables (e.g. sector size) and
so we need to encode both the table and the option that conflicts.

Add support for a two dimensional conflict definition and convert
all the code over to use it.

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 years agomkfs: protofile only needs to be set up once
Dave Chinner [Wed, 3 Jan 2018 20:10:10 +0000 (14:10 -0600)] 
mkfs: protofile only needs to be set up once

Signed-Off-By: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>