thirdparty/xfsprogs-dev.git
7 weeks agoxfsprogs: Release v5.0.0 master v5.0.0
Eric Sandeen [Fri, 3 May 2019 17:14:36 +0000 (12:14 -0500)]
xfsprogs: Release v5.0.0

Update all the necessary files for a 5.0.0 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 weeks agoxfs_db: scan entire file system when using 'frag'
Jorge Guerra [Fri, 3 May 2019 16:52:32 +0000 (11:52 -0500)]
xfs_db: scan entire file system when using 'frag'

While running the 'frag' command of 'xfs_db' we noticed that the
tool is not scanning all the files in the file system.  We noticed
this when we modified the tool to print the inodes of all the files
scanned.  For example:

 $ find /mnt/xfsdisk -type f | wc -l
 1782674
 $ xfs_db -r -c frag /dev/sdXX  | grep MB | awk '{print $5}' | paste -s -d+ | bc
 656818

Upon inspecting the code we noticed that the scanfunc_ino function
stops processing a given inode block once it encounters a free leaf.
However, in practice we see that inodes are necessarily always layed
out contiguously on the leaf node.  This resulted in the 'frag'
command skipping some valid inodes.

In this change we modify the scanfunc_ino function to skip freed
inodes.  With the change in place we ran the same experiment again
and noticed a more accurate file count:

 $ find /mnt/d0 -type f | wc -l
 1810442
 $ xfs_db -r -c frag /dev/sdXX  | grep MB | awk '{print $5}' | paste -s -d+ | bc
 1810442

Fixes: 2a5eb70c ("xfs_db: teach the frag command about sparse inode chunks")
Signed-off-by: Jorge Guerra <jorgeguerra@fb.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
7 weeks agoxfs_io: rework includes for statx structures
Eric Sandeen [Fri, 3 May 2019 16:51:37 +0000 (11:51 -0500)]
xfs_io: rework includes for statx structures

Only include the kernel's linux/stat.h headers if we haven't
already picked up statx bits from glibc, to avoid redefinition.

Suggested-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Tested-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfsprogs: Release v5.0.0-rc1 v5.0.0-rc1
Eric Sandeen [Fri, 26 Apr 2019 21:50:48 +0000 (16:50 -0500)]
xfsprogs: Release v5.0.0-rc1

Update all the necessary files for a 5.0.0-rc1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: refactor manage_zones()
Eric Sandeen [Fri, 26 Apr 2019 21:50:48 +0000 (16:50 -0500)]
libxfs: refactor manage_zones()

It's bizarre to have manage_zones() both set up and tear down zones.
It's even more bizarre to have xfs_dir_startup() buried in there.

Refactor init/destory into 2 functions, and call xfs_dir_startup()
separately from zone init similar to what kernelspace does.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibfrog: fix memory leak in bitmap_free
Darrick J. Wong [Fri, 26 Apr 2019 21:50:48 +0000 (16:50 -0500)]
libfrog: fix memory leak in bitmap_free

Free the bitmap struct before we null out the caller's pointer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: shorten inode item lifetime
Darrick J. Wong [Fri, 26 Apr 2019 21:50:48 +0000 (16:50 -0500)]
libxfs: shorten inode item lifetime

Shorten the inode item lifetime so that we only keep them around while
the inode is joined with a transaction.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: fix buffer log item lifetime weirdness
Darrick J. Wong [Fri, 26 Apr 2019 21:50:48 +0000 (16:50 -0500)]
libxfs: fix buffer log item lifetime weirdness

In xfsprogs, the lifetime of xfs_buf log items doesn't match the kernel
because we keep them around after comitting or cancelling transactions.
This is confusing, so change the lifetime to be consistent.  Worse yet,
if an inode cluster buffer gets bjoined to a transaction (e.g. someone
called xfs_trans_read_buf) we'll leak it when flushing an inode core
back to that buffer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: don't touch buffer log item pointer when flushing inode log item
Darrick J. Wong [Fri, 26 Apr 2019 21:50:48 +0000 (16:50 -0500)]
libxfs: don't touch buffer log item pointer when flushing inode log item

When we're flushing an inode log item, it is not necessary to mess with
the inode cluster buffer's log item because the iflush code paths pass
the inode log item directly.  The unconditional reset causes us to leak
buffer log items.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: refactor buffer item release code
Darrick J. Wong [Fri, 26 Apr 2019 21:50:10 +0000 (16:50 -0500)]
libxfs: refactor buffer item release code

Refactor the buffer item release code into a helper, which we will use
in subsequent patches to make the buffer log item lifetime match the
kernel equivalents.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agomisc: fix strncpy length complaints
Darrick J. Wong [Fri, 26 Apr 2019 20:44:21 +0000 (15:44 -0500)]
misc: fix strncpy length complaints

Fix a number of complaints about feeding sizeof(dest) directly to
strncpy.  We do this by feeding strncpy the length of the buffer minus
one, having checked that the allocated space are long enough.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: drop the ifork_ops parameter from libxfs_inode_verify_forks
Darrick J. Wong [Fri, 26 Apr 2019 20:42:00 +0000 (15:42 -0500)]
libxfs: drop the ifork_ops parameter from libxfs_inode_verify_forks

Now that the inode remembers its own ifork_ops, we can drop the second
parameter from libxfs_inode_verify_forks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: retain ifork_ops when flushing inode
Darrick J. Wong [Fri, 26 Apr 2019 20:41:59 +0000 (15:41 -0500)]
libxfs: retain ifork_ops when flushing inode

Retain the ifork ops used to validate the inode so that we can use the
same one to iflush it.  xfs_repair phase 6 can use multiple transactions
to fix various inode problems, which means that the inode might not be
fully fixed when each transaction commits.

This can be a particular problem if there's a shortform directory with
both invalid directory entries and incorrect i8count.  Phase 3 will set
the parent inode to "0" to signal to phase 6 that it needs to reset the
parent and i8count, but phase 6 starts a transaction to junk the bad
entries which fail to commit because the parent is invalid:

fixing i8count in inode 69022994673
Invalid inode number 0x0
xfs_dir_ino_validate: XFS_ERROR_REPORT
Metadata corruption detected at 0x464eb0, inode 0x10121750f1 data fork
xfs_repair: warning - iflush_int failed (-117)

And thus the inode fixes never get written out.

Reported-by: Arkadiusz Miskiewicz <arekm@maven.pl>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: correctly account for free space btree shrinks when fixing freelist
Darrick J. Wong [Fri, 26 Apr 2019 20:41:59 +0000 (15:41 -0500)]
xfs_repair: correctly account for free space btree shrinks when fixing freelist

When we fix the freelist at the end of build_agf_agfl in phase 5 of
repair, we need to create incore rmap records for the blocks that get
added to the AGFL.  We can't let the regular freelist fixing code use
the regular on-disk rmapbt update code because the rmapbt isn't fully
set up yet.

Unfortunately, the original code fails to account for the fact that the
free space btrees can shrink when we allocate blocks to fix the
freelist; those blocks are also put on the freelist, but there are
already incore rmaps for all the free space btree blocks.  We must not
create (redundant) incore rmaps for those blocks.  If we do, repair
fails with a complaint that rebuilding the rmapbt failed during phase 5.
xfs/137 on a 1k block size occasionally triggers this bug.

To fix the problem, construct a bitmap of all OWN_AG blocks that we know
about before traversing the AGFL, and only create new incore rmaps for
those AGFL blocks that are not already tracked in the bitmap.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_info: use findmnt to handle mounted block devices
Darrick J. Wong [Fri, 26 Apr 2019 20:41:59 +0000 (15:41 -0500)]
xfs_info: use findmnt to handle mounted block devices

Use findmnt to determine if the passed-in argument is associated with a
mount point, and if so, use spaceman to query the mounted filesystem.
If the user passed in a file, try to find out if it's a loop mounted
live filesystem and if so query the live filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoscrub: fix Makefile targets which depend on builddefs
Darrick J. Wong [Fri, 26 Apr 2019 20:41:59 +0000 (15:41 -0500)]
scrub: fix Makefile targets which depend on builddefs

Add Makefile dependencies for targets that require variables set in
builddefs.  Although most of the required variables are file paths
defined during the ./configure process, we cannot simply use
AC_CONFIG_FILES to generate the scripts because that macro only expands
one level deep and its documentation says that it's only to be used for
generating makefiles, not build targets themselves.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfsprogs: remove unused function prototypes
Eric Sandeen [Fri, 26 Apr 2019 20:41:12 +0000 (15:41 -0500)]
xfsprogs: remove unused function prototypes

These are prototypes for functions that aren't defined anywhere in
the code, so remove them.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_scrub: remove redundant function declarations
Darrick J. Wong [Fri, 26 Apr 2019 20:40:51 +0000 (15:40 -0500)]
xfs_scrub: remove redundant function declarations

Remove a redundant function declaration.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agomkfs: don't use DIFLAG values for fsx_xflags
Darrick J. Wong [Fri, 26 Apr 2019 20:40:46 +0000 (15:40 -0500)]
mkfs: don't use DIFLAG values for fsx_xflags

mkfs shouldn't be using DIFLAG values where FS_XFLAG values are
required.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: minor adjustments for previous patch mods]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_scrub: remove pointless xfs_verify_error_info struct
Darrick J. Wong [Fri, 26 Apr 2019 20:40:42 +0000 (15:40 -0500)]
xfs_scrub: remove pointless xfs_verify_error_info struct

The xfs_verify_error_info structure is a strict subset of
media_verify_info so just pass that around.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: rename another variable to match structure]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_scrub: rename confusing structure
Darrick J. Wong [Fri, 26 Apr 2019 20:40:37 +0000 (15:40 -0500)]
xfs_scrub: rename confusing structure

Rename xfs_verify_extent to media_verify_state so that the name more
accurately reflects what we're storing in the structure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_scrub: remove xfs_ prefixes from structure names
Darrick J. Wong [Fri, 26 Apr 2019 20:40:33 +0000 (15:40 -0500)]
xfs_scrub: remove xfs_ prefixes from structure names

Drop the xfs_ prefix from scrub's struct definitions since they're
private to the program.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_db: metadump should handle symlinks properly
Darrick J. Wong [Fri, 26 Apr 2019 20:40:28 +0000 (15:40 -0500)]
xfs_db: metadump should handle symlinks properly

Remote symlink target blocks are multi-fsb objects on XFS v5 filesystems
because we only write one rmt header per data fork extent.  For fs
blocksize >= 2048 we never have more than one block and therefore nobody
noticed, but for blocksize == 1024 this is definitely not true and leads
to metadump spraying error messages about symlink block crc errors.
Therefore, reformulate the symlink metadump into a multi-fsb dump
function.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: shrink the map declaration]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_db: refactor multi-fsb object detection decision making
Darrick J. Wong [Fri, 26 Apr 2019 20:40:24 +0000 (15:40 -0500)]
xfs_db: refactor multi-fsb object detection decision making

Pull the "is this a multi-fsb object" decision into a separate function
that we can keep close to the actual multi-fsb object dispatcher.  We
will soon make the machinery more complex so we do this to avoid having
a big hairy if statement.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_db: refactor metadump handling of multi-fsb objects
Darrick J. Wong [Fri, 26 Apr 2019 20:40:19 +0000 (15:40 -0500)]
xfs_db: refactor metadump handling of multi-fsb objects

Separate the multi-fsb object dispatch from actual dir block processing
so that we can implement symlink handling correctly as a multi-fsb file.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: better cli option parameter checking
Darrick J. Wong [Fri, 26 Apr 2019 20:40:14 +0000 (15:40 -0500)]
xfs_repair: better cli option parameter checking

For the xfs_repair options that require a parameter, check that the user
actually provided one instead of segfaulting.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_io: fix label parsing and validation
Darrick J. Wong [Fri, 26 Apr 2019 20:40:10 +0000 (15:40 -0500)]
xfs_io: fix label parsing and validation

When we're trying to set a new label, check the length to make sure we
won't overflow the label size, and size label[] so that we can use
strncpy without static checker complaints.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_io: don't read garbage stack contents if INUMBERS goes nuts
Darrick J. Wong [Fri, 26 Apr 2019 20:40:05 +0000 (15:40 -0500)]
xfs_io: don't read garbage stack contents if INUMBERS goes nuts

In theory INUMBERS will never return an ocount of zero, but on the off
chance it ever does we'll negative index the igroup array and return
stack contents for an inode number.  Don't do that.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_scrub: include unicrash.h in unicrash.c
Darrick J. Wong [Fri, 26 Apr 2019 20:40:00 +0000 (15:40 -0500)]
xfs_scrub: include unicrash.h in unicrash.c

Fix broken function prototype checking in unicrash.c by actually
including its header.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: remove libxfs_trans_iget
Darrick J. Wong [Fri, 26 Apr 2019 20:39:56 +0000 (15:39 -0500)]
libxfs: remove libxfs_trans_iget

libxfs_trans_iget no longer has a counterpart in the kernel.  Remove it
and make the xfs_iget/xfs_trans_ijoin usage consistent throughout
xfsprogs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibxfs: refactor the open-coded libxfs_trans_bjoin calls
Darrick J. Wong [Fri, 26 Apr 2019 20:39:51 +0000 (15:39 -0500)]
libxfs: refactor the open-coded libxfs_trans_bjoin calls

Refactor open-coded bjoin code to use libxfs_trans_bjoin.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agolibfrog: hoist bitmap out of scrub
Darrick J. Wong [Fri, 26 Apr 2019 20:39:46 +0000 (15:39 -0500)]
libfrog: hoist bitmap out of scrub

Move the bitmap code to libfrog so that we can use bitmaps in
xfs_repair.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: pass ops through during scan
Darrick J. Wong [Fri, 26 Apr 2019 20:39:41 +0000 (15:39 -0500)]
xfs_repair: pass ops through during scan

Pass the buffer ops through scan_sbtree so that we detect finobt blocks
properly and we don't have to keep switching on magic numbers for the
free space btrees.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: refactor buffer ops assignments during phase 5
Darrick J. Wong [Fri, 26 Apr 2019 20:39:35 +0000 (15:39 -0500)]
xfs_repair: refactor buffer ops assignments during phase 5

Refactor the buffer ops assignments in phase 5 to use a helper function
to determine the correct buf_ops instead of open-coding them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: fix uninitialized variable warnings
Darrick J. Wong [Fri, 26 Apr 2019 20:39:29 +0000 (15:39 -0500)]
xfs_repair: fix uninitialized variable warnings

Fix some uninitialized variable warnings because ASSERT disappears if
DEBUG is not defined.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: bump the irec on-disk nlink when adding lost+found
Darrick J. Wong [Fri, 26 Apr 2019 20:39:23 +0000 (15:39 -0500)]
xfs_repair: bump the irec on-disk nlink when adding lost+found

We increment the nlink of the root directory inode when creating a
"lost+found" directory during phase 6, but we don't update the irec copy
of the root dir nlink.  This normally gets papered over by phase 7, but
this can fail badly if:

1) The root directory had an entry to a busted subdirectory, so
   that root directory will have nlink == 3, but in the ino_tree,
   counted_nlinks == 2 and disk_nlinks == 3.

2) Phase 6 creates lost+found to root the files that were in the busted
   directory, we'll set nlink = 4 and counted_nlinks = 3.  The correct
   nlink is 3 ('.', '..', 'lost+found'), not 4.

3) During phase 7, we see that counted_nlinks == disk_nlinks and so we
   totally fail to correct the on-disk inode.

4) A subsequent run of xfs_repair complains about the nlink being 4
   instead of 3.

To fix this, we have to adjust the irec's disk_nlinks in step 2 so that
phase 7 seeds that counted_nlinks < disk_nlinks and resets nlink to
counted_nlinks.  This can be reproduced somewhat frequently by xfs/117.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
8 weeks agoxfs_repair: reinitialize the root directory nlink correctly
Darrick J. Wong [Fri, 26 Apr 2019 20:39:17 +0000 (15:39 -0500)]
xfs_repair: reinitialize the root directory nlink correctly

In mk_root_dir, we reinitialize the root directory inode with a link
count of 1.  This differs from mkfs parseproto, which initializes the
root to have a link count of 2.  The nlink discrepancy in repair is
caught and corrected during phase 7, but this is unnecessary since we
should set it properly in the first place.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agomkfs: validate extent size hint parameters
Darrick J. Wong [Thu, 18 Apr 2019 18:19:39 +0000 (13:19 -0500)]
mkfs: validate extent size hint parameters

Validate extent and cow extent size hints that are passed to mkfs so
that we avoid formatting a filesystem that will never mount.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: use DIFLAG macros for now to be consistent, fixed later]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: fix typo in unicrash header file
Eric Sandeen [Thu, 18 Apr 2019 18:19:39 +0000 (13:19 -0500)]
xfs_scrub: fix typo in unicrash header file

The no-op #definintion was missing 'fs_' so remained undefined
in the #else case.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_db: use TYP_FINOBT for finobt metadump
Darrick J. Wong [Thu, 28 Mar 2019 23:07:23 +0000 (18:07 -0500)]
xfs_db: use TYP_FINOBT for finobt metadump

Use the correct xfs_db type for dumping free inode btree blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: check label for misleading characters
Darrick J. Wong [Thu, 28 Mar 2019 23:05:02 +0000 (18:05 -0500)]
xfs_scrub: check label for misleading characters

Make sure that we can retrieve the label and that it doesn't contain
anything potentially misleading.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: don't close mnt_fd when mnt_fd open fails
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub: don't close mnt_fd when mnt_fd open fails

If we fail to open the mountpoint during phase 1 of scrub, don't bother
trying to close the file descriptor since it's silly to spray error
messages about that.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: one read/verify pool per disk
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub: one read/verify pool per disk

Simplify the read/verify pool code further by creating one pool per
disk.  This enables us to tailor the concurrency levels of each disk to
that specific disk so that if we have a mixed hdd/ssd environment we
don't flood the hdd with a lot of requests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: don't expose internal pool state
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub: don't expose internal pool state

In xfs_scrub, the read/verify pool tries to coalesce the media
verification requests into a smaller number of large IOs.  There's no
need to force callers to keep track of this internal state, so just move
all that into read_verify.c.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: use datadev parallelization estimates for thread count
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub: use datadev parallelization estimates for thread count

During phases 2-5, xfs_scrub should estimate the level of
parallelization possible on the data device to determine the number of
threads spawned to scrub filesystem metadata, not just blindly using the
number of CPUs.  This avoids flooding non-rotational storage with random
reads, which totally destroys performance and makes scrub runtimes
higher.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub: rename the global nr_threads
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub: rename the global nr_threads

Various functions have nr_threads local variables that shadow the global
one.  Since the global one forces the number of threads we use, change
its name to remove this ambiguity and reflect what it really does.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub_all.timer: activate after most of the system is up
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub_all.timer: activate after most of the system is up

We really don't want the xfs_scrub_all timer triggering while the system
is booting up because not all the mounts will have finished, networking
might not be up for reporting, and slowing down bootup annoys people.
Therefore, delay the xfs_scrub_all service's activation until after the
system has started all the big pieces it's going to start.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_scrub_all: walk the lsblk device/fs hierarchy correctly
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_scrub_all: walk the lsblk device/fs hierarchy correctly

Back when I was designing xfs_scrub_all, I na├»vely assumed that the
emitted output would always list physical storage before the virtual
devices stacked atop it.  However, this is not actually true when one
omits the "NAME" column, which is crucial to forcing the output (json or
otherwise) to capture the block device hierarchy.  If the assumption is
violated, the program crashes with a python exception.

To fix this, force the hierarchal json output and restructure the
discovery routines to walk the json object that we receive, from the top
(physical devices) downwards to wherever there are live xfs filesystems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_io: document fzero_f -k option in manpage
Eric Sandeen [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_io: document fzero_f -k option in manpage

Perhaps the reason -k was broken is that nobody knew it existed?

Fixes: 938904c4 ("xfs_io: add fzero command for zeroing range via fallocate")
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_io: don't walk off the end of argv in fzero_f
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_io: don't walk off the end of argv in fzero_f

The fzero_f function doesn't check that there are enough non-switch
parameters to supply offset and length arguments to fallocate.  As a
result, we can walk off the end of the argv array and crash.  A
secondary problem is that we don't use getopt to detect the -k, which is
not how most xfs_io commands work.

Therefore, use getopt to detect the -k argument and rewire the offset
and length interpretation code to check optind and use argv correctly.
This bug is trivially reproduced by "xfs_io -c 'fzero -k 0' /some/file".

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_io: statx -r should print attributes_mask
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_io: statx -r should print attributes_mask

We're dumping the raw structure, so we ought to dump everything,
including the attributes_mask field.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoxfs_io: actually check copy file range helper return values
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
xfs_io: actually check copy file range helper return values

We need to check the return value of copy_src_filesize and
copy_dst_truncate because either could return -1 due to fstat/ftruncate
failure.

Fixes: 628e112afdd98c5 ("xfs_io: implement 'copy_range' command")
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agodebian: enable parallel make
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
debian: enable parallel make

Use parallel make to speed up dpkg builds.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Nathan Scott <nathans@debian.org>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agodebian: don't bypass top level Makefile when building subdirs
Darrick J. Wong [Thu, 28 Mar 2019 23:05:01 +0000 (18:05 -0500)]
debian: don't bypass top level Makefile when building subdirs

The top level Makefile does some processing to set build environment
variables (Q and CHECK_CMD).  debian/rules uses -C to build subdirs
directly, which bypases this feature of the top-level makefile, which
causes more build spew than necessary (because Q never gets set to quiet
the build).

Since the top level makefile can be used to build the subdirs
debian/rules cares about, drop the -C and build subdirs via the top
level Makefile to quiet the build.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Nathan Scott <nathans@debian.org>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agodebian: drop dangling libhandle.a symlinks in xfslibs-dev
Darrick J. Wong [Thu, 28 Mar 2019 23:04:51 +0000 (18:04 -0500)]
debian: drop dangling libhandle.a symlinks in xfslibs-dev

We don't ship static libhandle libraries anymore, so make sure we drop
the symlink.

Fixes: ec1cf08dbeb2d ("Several updates to use more modern Debian packaging")
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Nathan Scott <nathans@debian.org>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoconfigure.ac: fix alignment of features
Darrick J. Wong [Thu, 28 Mar 2019 23:04:51 +0000 (18:04 -0500)]
configure.ac: fix alignment of features

Fix the alignment of the feature options in the --help screen.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agoconfigure: use sys/xattr.h for fsetxattr detection
Darrick J. Wong [Thu, 28 Mar 2019 23:04:51 +0000 (18:04 -0500)]
configure: use sys/xattr.h for fsetxattr detection

The only user of fsetxattr and HAVE_FSETXATTR is fsr, which includes
sys/xattr.h (from libc).  However, the m4 macro to detect fsetxattr
support requires attr/xattr.h (from libattr).  libattr dropped xattr.h
last year, so update the check.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2 months agolibxfs: fix repair deadlock due to failed inode flushes.
Dave Chinner [Thu, 28 Mar 2019 23:04:51 +0000 (18:04 -0500)]
libxfs: fix repair deadlock due to failed inode flushes.

If inode_item_done() fails to flush an inode after we've grabbed a
reference to the underlying buffer during a transaction commit, we
fail to put the buffer and hence leak it. We then deadlock on the
next lookup ofthe inode buffer as it is still locked and no-one owns
it.

To fix it, put the buffer on error so that it gets unlocked and
can be recovered appropriately in a later phase of repair.

Reported-by: Arkadiusz Miskiewicz <arekm@maven.pl>
Fixes: d15188a1ec14 ("xfs: rework the inline directory verifiers")
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: stringify scrub types in ftrace output libxfs-5.0-sync
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: stringify scrub types in ftrace output

Source kernel commit: 86d163dbfe2ac0b30fbb6e256301abbfa9e4549e

Use __print_symbolic to print the scrub type in ftrace output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: stringify btree cursor types in ftrace output
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: stringify btree cursor types in ftrace output

Source kernel commit: c494213f30080423b70b24b6af7f6da554d9390f

Use __print_symbolic to print the btree type in ftrace output.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: move XFS_INODE_FORMAT_STR mappings to libxfs
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: move XFS_INODE_FORMAT_STR mappings to libxfs

Source kernel commit: 0357d21a6c9be2870904598b4767c7d424524849

Move XFS_INODE_FORMAT_STR to libxfs so that we don't forget to keep it
updated, and add necessary TRACE_DEFINE_ENUM.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: move XFS_AG_BTREE_CMP_FORMAT_STR mappings to libxfs
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: move XFS_AG_BTREE_CMP_FORMAT_STR mappings to libxfs

Source kernel commit: 05c753c4cf53f51a7e35fcfe684500113cf1fd13

Move XFS_AG_BTREE_CMP_FORMAT_STR to libxfs so that we don't forget to
keep it updated, and TRACE_DEFINE_ENUM the values while we're at it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: fix symbolic enum printing in ftrace output
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: fix symbolic enum printing in ftrace output

Source kernel commit: 85f8dff00a3193fe5659aa4c91adde31723c0d3d

ftrace's __print_symbolic() has a (very poorly documented) requirement
that any enum values used in the symbol to string translation table be
wrapped in a TRACE_DEFINE_ENUM so that the enum value can be encoded in
the ftrace ring buffer.  Fix this unsatisfied requirement.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: cache minimum realtime summary level
Omar Sandoval [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: cache minimum realtime summary level

Source kernel commit: 355e3532132b487ebf6a4900fad8f3525fa3e137

The realtime summary is a two-dimensional array on disk, effectively:

u32 rsum[log2(number of realtime extents) + 1][number of blocks in the bitmap]

rsum[log][bbno] is the number of extents of size 2**log which start in
bitmap block bbno.

xfs_rtallocate_extent_near() uses xfs_rtany_summary() to check whether
rsum[log][bbno] != 0 for any log level. However, the summary array is
stored in row-major order (i.e., like an array in C), so all of these
entries are not adjacent, but rather spread across the entire summary
file. In the worst case (a full bitmap block), xfs_rtany_summary() has
to check every level.

This means that on a moderately-used realtime device, an allocation will
waste a lot of time finding, reading, and releasing buffers for the
realtime summary. In particular, one of our storage services (which runs
on servers with 8 very slow CPUs and 15 8 TB XFS realtime filesystems)
spends almost 5% of its CPU cycles in xfs_rtbuf_get() and
xfs_trans_brelse() called from xfs_rtany_summary().

One solution would be to also store the summary with the dimensions
swapped. However, this would require a disk format change to a very old
component of XFS.

Instead, we can cache the minimum size which contains any extents. We do
so lazily; rather than guaranteeing that the cache contains the precise
minimum, it always contains a loose lower bound which we tighten when we
read or update a summary block. This only uses a few kilobytes of memory
and is already serialized via the realtime bitmap and summary inode
locks, so the cost is minimal. With this change, the same workload only
spends 0.2% of its CPU cycles in the realtime allocator.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: precalculate cluster alignment in inodes and blocks
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: precalculate cluster alignment in inodes and blocks

Source kernel commit: c1b4a321ede083521b91c314e1c4fa233ac33740

Store the inode cluster alignment information in units of inodes and
blocks in the mount data so that we don't have to keep recalculating
them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: precalculate inodes and blocks per inode cluster
Darrick J. Wong [Wed, 27 Feb 2019 23:13:45 +0000 (17:13 -0600)]
xfs: precalculate inodes and blocks per inode cluster

Source kernel commit: 83dcdb4469e759f984db92616d7885fc14329841

Store the number of inodes and blocks per inode cluster in the mount
data so that we don't have to keep recalculating them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: add a block to inode count converter
Darrick J. Wong [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: add a block to inode count converter

Source kernel commit: 43004b2a8da2652b5ec526269a8acfba7d3d219c

Add new helpers to convert units of fs blocks into inodes, and AG blocks
into AG inodes, respectively.  Convert all the open-coded conversions
and XFS_OFFBNO_TO_AGINO(, , 0) calls to use them, as appropriate.  The
OFFBNO_TO_AGINO macro is retained for xfs_repair.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: remove xfs_rmap_ag_owner and friends
Darrick J. Wong [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: remove xfs_rmap_ag_owner and friends

Source kernel commit: 7280fedaf3a0f9097c0621c7d5b35849954d7f54

Owner information for static fs metadata can be defined readonly at
build time because it never changes across filesystems.  This enables us
to reduce stack usage (particularly in scrub) because we can use the
statically defined oinfo structures.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: const-ify xfs_owner_info arguments
Darrick J. Wong [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: const-ify xfs_owner_info arguments

Source kernel commit: 66e3237e724c6650dca03627b40bb00a812d3f7a

Only certain functions actually change the contents of an
xfs_owner_info; the rest can accept a const struct pointer.  This will
enable us to save stack space by hoisting static owner info types to
be const global variables.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: streamline defer op type handling
Darrick J. Wong [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: streamline defer op type handling

Source kernel commit: 02b100fb83f9b0f8719deef6c4ed973b4d9ce00c

There's no need to bundle a pointer to the defer op type into the defer
op control structure.  Instead, store the defer op type enum, which
enables us to shorten some of the lines.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: idiotproof defer op type configuration
Darrick J. Wong [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: idiotproof defer op type configuration

Source kernel commit: bc9f2b7c8a732d896753709cc9d495780ba7e9f9

Recently, we forgot to port a new defer op type to xfsprogs, which
caused us some userspace pain.  Reorganize the way we make libxfs
clients supply defer op type information so that all type information
has to be provided at build time instead of risky runtime dynamic
configuration.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: zero length symlinks are not valid
Dave Chinner [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: zero length symlinks are not valid

Source kernel commit: 43feeea88c9cb2955b9f7ba8152ec5abeea42810

A log recovery failure has been reproduced where a symlink inode has
a zero length in extent form. It was caused by a shutdown during a
combined fstress+fsmark workload.

The underlying problem is the issue in xfs_inactive_symlink(): the
inode is unlocked between the symlink inactivation/truncation and
the inode being freed. This opens a window for the inode to be
written to disk before it xfs_ifree() removes it from the unlinked
list, marks it free in the inobt and zeros the mode.

For shortform inodes, the fix is simple. xfs_ifree() clears the data
fork state, so there's no need to do it in xfs_inactive_symlink().
This means the shortform fork verifier will not see a zero length
data fork as it mirrors the inode size through to xfs_ifree()), and
hence if the inode gets written back and the fork verifiers are run
they will still see a fork that matches the on-disk inode size.

For extent form (remote) symlinks, it is a little more tricky. Here
we explicitly set the inode size to zero, so the above race can lead
to zero length symlinks on disk. Because the inode is unlinked at
this point (i.e. on the unlinked list) and unreferenced, it can
never be seen again by a user. Hence when we set the inode size to
zeor, also change the type to S_IFREG. xfs_ifree() expects S_IFREG
inodes to be of zero length, and so this avoids all the problems of
zero length symlinks ever hitting the disk. It also avoids the
problem of needing to handle zero length symlink inodes in log
recovery to replay the extent free intents and the remaining
deferops to free the extents the symlink used.

Also add a couple of asserts to warn us if zero length symlinks end
up in either the symlink create or inactivation paths.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
3 months agoxfs: libxfs: move xfs_perag_put late
Pan Bian [Wed, 27 Feb 2019 23:13:44 +0000 (17:13 -0600)]
xfs: libxfs: move xfs_perag_put late

Source kernel commit: fe5ed6c22e94b131ed5608d66ebce1efc39a7edb

The function xfs_alloc_get_freelist calls xfs_perag_put to drop the
reference. However, pag->pagf_btreeblks is read and written after the
put operation. This patch moves the put operation later.

Signed-off-by: Pan Bian <bianpan2016@163.com>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
[darrick: minor changelog edits]
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfsprogs: Release v4.20.0 v4.20.0
Eric Sandeen [Fri, 22 Feb 2019 18:07:00 +0000 (12:07 -0600)]
xfsprogs: Release v4.20.0

Update all the necessary files for a 4.20.0 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfsprogs: Release v4.20.0-rc1 v4.20.0-rc1
Eric Sandeen [Fri, 8 Feb 2019 19:28:49 +0000 (13:28 -0600)]
xfsprogs: Release v4.20.0-rc1

Update all the necessary files for a 4.20.0-rc1 release.

Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_repair: fix incorrect return value in namecheck()
Eric Sandeen [Fri, 1 Feb 2019 17:54:15 +0000 (11:54 -0600)]
xfs_repair: fix incorrect return value in namecheck()

Obviously a directory entry with a '/' in the name should return
1, i.e. failure.  This was just a dumb thinko.

Fixes: 45571fd5885d ("xfs_repair: allow '/' in attribute names")

Reported-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfsprogs: Remove barrier/nobarrier mount options from xfs.5
Eric Sandeen [Tue, 29 Jan 2019 17:01:22 +0000 (11:01 -0600)]
xfsprogs: Remove barrier/nobarrier mount options from xfs.5

Remove the now-removed barrier/nobarrier mount options from
the xfs.5 manpage.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_db: add -d to short help for write command
Eric Sandeen [Mon, 28 Jan 2019 22:11:19 +0000 (16:11 -0600)]
xfs_db: add -d to short help for write command

And note in the man page that -c and -d are exclusive.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Bill O'Donnell <billodo@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfsprogs: remove generated scrub files under clean target
Eric Sandeen [Mon, 28 Jan 2019 22:09:45 +0000 (16:09 -0600)]
xfsprogs: remove generated scrub files under clean target

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_repair: initialize non-leaf finobt blocks with correct magic
Brian Foster [Mon, 28 Jan 2019 22:05:16 +0000 (16:05 -0600)]
xfs_repair: initialize non-leaf finobt blocks with correct magic

The free inode btree construction code in xfs_repair has a bug where
any non-leaf nodes outside of the leftmost block at the associated
level in the tree are incorrectly initialized with the inobt magic
value. Update the prop_ino_cursor() path responsible for growing the
non-leaf portion of the inode btrees to use the btnum of the
specific tree being generated rather than the hardcoded inode btree
type.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reported-by: Lucas Stach <l.stach@pengutronix.de>
Root-caused-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_db: fix finobt record decoding when sparse inodes enabled
Darrick J. Wong [Mon, 28 Jan 2019 19:03:15 +0000 (13:03 -0600)]
xfs_db: fix finobt record decoding when sparse inodes enabled

Use the sparse inobt record field decoder (inobt_spcrc_hfld) to decode
finobt records when sparse inodes are enabled.  Otherwise, xfs_db
prints out bogus things like:

recs[1] = [startino,freecount,free]
1:[214720,16429,0xfffffffffff80000]

There can never be 16429 records in an inode btree record; instead it
should print:

recs[1] = [startino,holemask,count,freecount,free]
1:[214720,0,64,45,0xfffffffffff80000]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_repair: fix off by one error when rebuilding high keys
Darrick J. Wong [Mon, 28 Jan 2019 19:03:11 +0000 (13:03 -0600)]
xfs_repair: fix off by one error when rebuilding high keys

Fix an off-by-one error when scanning a rmap btree block for high keys
as part of rebuilding rmap btrees during phase 5.  This causes
xfs_repair to emit a corrupt filesystem, which is bad.

This can be reproduced pretty easily by exporting
TEST_XFS_REPAIR_REBUILD=1 and running generic/051 with a 1k block size.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_quota: fix false error reporting of project inheritance flag is not set
Achilles Gaikwad [Mon, 28 Jan 2019 19:03:08 +0000 (13:03 -0600)]
xfs_quota: fix false error reporting of project inheritance flag is not set

After kernel commit:

9336e3a7 "xfs: project id inheritance is a directory only flag"

xfs stopped setting the project inheritance flag on regular files, but
userspace quota code still checks for it and will now issue the error:

"project inheritance flag is not set"

for every regular file during quotacheck.  Fix this by only checking
for the flag on directories.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1663502
Reported-by: Steven Gardner <sgardner@redhat.com>
Signed-off-by: Achilles Gaikwad <agaikwad@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
4 months agoxfs_repair: allow '/' in attribute names
Eric Sandeen [Mon, 28 Jan 2019 19:03:03 +0000 (13:03 -0600)]
xfs_repair: allow '/' in attribute names

For some reason, since the earliest days of XFS, a '/' character
in an extended attribute name has been treated as corruption by
xfs_repair.  This despite nothing in other userspace tools or the
kernel having this restriction.

My best guess is that this was an unintentional leftover from
common code between dirs & attrs in the "da" code, and there has
never been a good reason for it.

Since userspace and kernelspace allow such a name to be set,
listed, and read, it seems wrong to flag it as corruption.
So, make this test conditional on whether we're validating a name
in a dir, as opposed to the name of an attr.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
5 months agoxfs_metadump: Zap dev inodes
Stefan Ring [Fri, 11 Jan 2019 22:58:17 +0000 (16:58 -0600)]
xfs_metadump: Zap dev inodes

Signed-off-by: Stefan Ring <stefanrin@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
5 months agoxfs_metadump: Zap unused space in inode btrees
Stefan Ring [Fri, 11 Jan 2019 22:57:51 +0000 (16:57 -0600)]
xfs_metadump: Zap unused space in inode btrees

Signed-off-by: Stefan Ring <stefanrin@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
5 months agoxfs_metadump: Zap freeindex blocks in directory inodes
Stefan Ring [Fri, 11 Jan 2019 22:57:27 +0000 (16:57 -0600)]
xfs_metadump: Zap freeindex blocks in directory inodes

Signed-off-by: Stefan Ring <stefanrin@gmail.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
5 months agoxfs_metadump: Zap multi fsb blocks
Stefan Ring [Fri, 11 Jan 2019 22:57:04 +0000 (16:57 -0600)]
xfs_metadump: Zap multi fsb blocks

Using basically the same code as in process_single_fsb_objects.

Signed-off-by: Stefan Ring <stefanrin@gmail.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
5 months agoxfs_metadump: Extend data zapping to XFS_DIR{2,3}_LEAFN_MAGIC blocks
Stefan Ring [Fri, 11 Jan 2019 22:56:37 +0000 (16:56 -0600)]
xfs_metadump: Extend data zapping to XFS_DIR{2,3}_LEAFN_MAGIC blocks

Signed-off-by: Stefan Ring <stefanrin@gmail.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_db: teach the frag command about sparse inode chunks
Eric Sandeen [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_db: teach the frag command about sparse inode chunks

This is the equivalent of:

ea8a48f xfs_check: process sparse inode chunks correctly

for the frag_f() command in xfs_db.

Without this, the xfs_db frag command shows corruption as it
wanders into blocks that are not inodes.

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=201823
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agomkfs.xfs: null-terminate symlinks created via protofile
Eric Sandeen [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
mkfs.xfs: null-terminate symlinks created via protofile

Now that we have a symlink verifier which checks that in-memory
symlink names are null-terminated, be sure we do that when we
create them via the mkfs protofile.

We only want to null-terminate inline data if it's a symlink;
we only ever /call/ newfile() with "dolocal" for symlinks, so
rename that function argument for clarity.

Then, rather than open-coding all this, just call
xfs_init_local_fork which handles it properly.

Zorro found this by running xfs/019 on an s390x machine, it
failed with:

 Metadata corruption detected at 0x101214a, inode 0x89 data fork

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reported-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfsprogs: don't install xfs_scrub man pages if config'd off
Eric Sandeen [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfsprogs: don't install xfs_scrub man pages if config'd off

Don't install man pages for xfs_scrub if it's turned off
in the config.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_io: copy_file_range length is a size_t
Dave Chinner [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_io: copy_file_range length is a size_t

copy_file_range() takes a size_t as it's length, not a "long long".
Therefore we need to be able to pass sizes larger than 8EB to it
to be able to test the interface fully and that requires copy_range
to accept all values except an explicit error value of "-1LL".

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Jan Tulak <jtulak@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_io: allow open file permissions to be changed
Dave Chinner [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_io: allow open file permissions to be changed

I need to be able to open a file read-write, then change the
permissions on the file to read-only to check that copy_file_range
returns EPERM correctly in that case. This can't be done as root,
because root ignores file permissions, but as a normal user we can't
open a 0444 file for writing and so can't actually test writing to
a read-only file without some method of "open read-write, change
permissions to read-only, try to write to file through open
read-write file".

So, allow adding or removing write permissions on an open file.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
[sandeen: Move man page entry to FILE section]
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoio: open pipes in non-blocking mode
Dave Chinner [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
io: open pipes in non-blocking mode

So that O_RDONLY open commands (such as from copy_range) do not
block forever waiting on a non-existent writer.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Jan Tulak <jtulak@redhat.com>
[sandeen: initialize st per djwong's suggestion]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_db: add missing string name for DBM_COWDATA
Darrick J. Wong [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_db: add missing string name for DBM_COWDATA

In db/check.c, typename[] is supposed to have strings for every DBM_
type, but we forgot one.  Add it now.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_scrub: move everything to /usr/sbin
Darrick J. Wong [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_scrub: move everything to /usr/sbin

Recently, it was pointed out that xfs_scrub{,_all} depend on components
and libraries (libicu, python) that live in /usr.  /sbin binaries
shouldn't depend on /usr, so let's move the scrub binaries to /usr/sbin.

Reported-by: xfs@tlinx.org
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_scrub: fix fractional reporting of single inodes
Darrick J. Wong [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_scrub: fix fractional reporting of single inodes

When there are fewer than 1024 inodes in the filesystem, scrub reports
fractional inodes in its final report:

35.2MiB data used;  5.0 inodes used.
34.2MiB data found; 5.0 inodes found.
5.0 inodes counted; 5.0 inodes checked.

Inodes are indivisible, so only report the fractional part when we have
a large enough number of inodes to perform a unit conversion:

35.2MiB data used;  5 inodes used.
34.2MiB data found; 5 inodes found.
5 inodes counted; 5 inodes checked.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_scrub: handle totally empty inode chunks
Darrick J. Wong [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_scrub: handle totally empty inode chunks

We try to load a single inobt record with each FSINUMBERS call.  If the
chunk is totally empty (which can happen when there are more than one
inobt record per block) we should skip to the next INUMBERS call since
there are no inodes to bulkstat.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
6 months agoxfs_repair: skip block reservation when fixing freelist
Darrick J. Wong [Wed, 12 Dec 2018 17:42:40 +0000 (11:42 -0600)]
xfs_repair: skip block reservation when fixing freelist

AGFL blocks are considered to be part of the fdblocks count, so there's
no need to obtain a block reservation when fixing the AGFL as part of
repair.  Asking for a reservation can cause repair to fail if the
superblock claims zero fdblocks because we haven't gotten far enough
into phase 5 to have reset the superblock counters.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>