Eric Sandeen [Wed, 29 Oct 2014 05:33:15 +0000 (16:33 +1100)]
libxfs: fix harmless sparse endian nit
h_crc is __le32 but cpu_to_be32() is... __be32. So sparse
complains, even though it's harmless.
Although sparse is smart about bare 0s, and we could
drop the swap, other places explicitly swap to keep
things clear (I guess?) so "swap" the 0 with the proper
routine.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 15 Sep 2014 23:23:45 +0000 (09:23 +1000)]
xfsprogs: add supported file attributes to xfs.5 manpage
The chattr(1) manpage suffers from the same problems mount(1) had:
many options listed, not kept up to date for various filesystems.
I've submitted a manpage update for chattr(1) which says to refer to
filesystem-specific manpages for supported attributes; this patch
updates xfs(5) to list the attributes supported by xfs.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 15 Sep 2014 23:23:40 +0000 (09:23 +1000)]
xfs_repair: preserve error state in process_shortform_attr
process_shortform_attr uses the "junkit" error to track whether an
error was found, but by assigning it directly to the result of
valuecheck, previous errors are ignored, leading to unrepairable
errors of the form i.e.
"entry has INCOMPLETE flag on in shortform attribute"
or
"entry contains illegal character in shortform attribute name"
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 15 Sep 2014 23:23:39 +0000 (09:23 +1000)]
xfs_repair: clear bad flags in process_dinode_int
process_dinode_int() reports bad flags if dino->di_flags &
~XFS_DIFLAG_ANY - i.e. if any flags are set outside the known set.
But then instead of clearing them, it does flags &= ~XFS_DIFLAG_ANY
which keeps *only* the bad flags. This leads to persistent,
unrepairable errors of the form:
"Bad flags set in inode XXX"
Fix this.
While we are at it, fix a couple lines which look like they used to
be continuation lines, but are no longer.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 15 Sep 2014 23:18:41 +0000 (09:18 +1000)]
xfs_fsr: fix leaks & catch error in fsrfile()
The allocated fshandlep leaks on most error paths; restructure with
an out: target that does all necessary freeing, and initialize
filehandles to -1 so that we know whether they need to be closed on
the error path.
While we're at it, if gettmpname() fails, we still return 0 for an
error, because error is initialized to 0 and only set otherwise by
fsrfile_common. So if gettmpname() fails, we return success from
the function even though we did no work. Fix that as well by
initializing error to -1.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Jan Kara [Mon, 15 Sep 2014 23:16:44 +0000 (09:16 +1000)]
repair: Set ftype for entries in lost+found
So far all entries in lost+found had file type XFS_DIR3_FT_UNKNOWN which
is somewhat annoying as the next xfs_repair pass will find these and
report as an error. Set proper file type when creating these entries.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Jan Tulak [Mon, 18 Aug 2014 01:11:06 +0000 (11:11 +1000)]
libhandle: fix installation for symlinked /usr
Canonicalize the pathnames for PKG_LIB_DIR and PKG_ROOT_LIB_DIR
before checking if they are the same. This is required for Fedora
which doesn't have a separate /usr/lib directory anymore.
[Christoph Hellwig: reformat and change to canonical names]
Reported-by: Jan Tulak <jan@tulak.me> Signed-off-by: Jan Tulak <jan@tulak.me> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Jan Kara [Fri, 18 Jul 2014 00:47:11 +0000 (10:47 +1000)]
logprint: Fix printing of AGF and AGI buffers
Currently xfs_logprint doesn't show detailed data about AGF and AGI
buffers and instead always shows "Out of space". This is because
xfs_agf_t has additional fields and padding which we never read from
disk and thus buffer length is always smaller than the size of
xfs_agf_t or xfs_agi_t respectively.
Fix the problem by only making sure we have enough data in the buffer
to contain all the information we want to print.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Jie Liu [Wed, 16 Jul 2014 03:54:47 +0000 (13:54 +1000)]
quota: fix NULL pointer dereference in report_f
Run xfs_quota report against an invalid XFS path without desired quota
limitation is enabled will hit SEGSEGV as fs_path is uninitialized, e.g.
# xfs_quota -xc 'report -up' /invalid_path
xfs_quota: cannot setup path for mount /invalid_path: No such file or directory
Segmentation fault (core dumped)
(gdb) r -xc 'report -up' /invalid_path
xfs_quota: cannot setup path for mount /invalid_path: No such file or directory
Program received signal SIGSEGV, Segmentation fault.
0x0000000000408b4d in report_f (argc=2, argv=0x105ea70) at report.c:627
627 else if (fs_path->fs_flags & FS_MOUNT_POINT)
This patch fixes report_f() to only do report if the fs_path is initialized.
Signed-off-by: Jie Liu <jeff.liu@oracle.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Wed, 16 Jul 2014 03:53:47 +0000 (13:53 +1000)]
libxcmd: make all comparisons using realpath'd paths
Both mountpoints and devices can be symlinks, so given a path
to look for, and mountpoints/devices from the system, use
realpath() on *everything* before making the comparison to see
if our path is a match.
So, with symlinks for mount points as well as for devices:
# xfs_quota -xc "report -h" /mnt/scratch2
User quota on /mnt/scratch (/dev/mapper/testvg-lvol0)
Blocks
User ID Used Soft Hard Warn/Grace
---------- ---------------------------------
root 0 0 0 00 [------]
# xfs_quota -xc "report -h" /mnt/scratch
User quota on /mnt/scratch (/dev/mapper/testvg-lvol0)
Blocks
User ID Used Soft Hard Warn/Grace
---------- ---------------------------------
root 0 0 0 00 [------]
# xfs_quota -xc "report -h" /dev/dm-3
User quota on /mnt/scratch (/dev/mapper/testvg-lvol0)
Blocks
User ID Used Soft Hard Warn/Grace
---------- ---------------------------------
root 0 0 0 00 [------]
# xfs_quota -xc "report -h" /dev/mapper/testvg-lvol0
User quota on /mnt/scratch (/dev/mapper/testvg-lvol0)
Blocks
User ID Used Soft Hard Warn/Grace
---------- ---------------------------------
root 0 0 0 00 [------]
The commit:
050a7f1 xfsprogs: handle symlinks etc in fs_table_initialise_mounts()
tried to fix this earlier, but only worked one way;
it compared the argument path in both given and realpath
form to the paths in getmntent, but did not compare to
the realpaths of the getmntent devices.
If we reduce everything, everywhere, to a realpath(), we've
got our best shot at finding the match.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Wed, 16 Jul 2014 03:52:47 +0000 (13:52 +1000)]
repair: copy, don't clear, stripe geometry in backup SB
Today, if we have a filesystem with stripe geometry and
a damaged primary superblock, we will zero out stripe geometry
if we have copied the backup.
I'm guessing this might be because changing geometry with mount
options only updates the primary, so backups aren't guaranteed
to be current or correct.
Unfortunately, that leaves us with sb 0 w/ no geom, and backups
*with* geom, so the next repair finds the mismatch, and complains.
(In other words, the 2nd repair does not come up clean.)_
And ... the second repair copies the backup stripe geometry back
into the primary!
Rather than clearing stripe geometry in this case, just leave it
at what was found in the backup super, and inform the user that this
was done. This leaves a consistent filesystem, and gives the user
a heads-up to double-check the result.
This can all be demonstrated and tested by running xfs/030 with
geometry set in MKFS_OPTIONS. (To really make the test pass,
we need to filter the warning out of repair output.)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Fri, 11 Jul 2014 02:00:51 +0000 (12:00 +1000)]
repair: handle uncorrected corruptions in phase 2
Some of the AG header corruptions detected by the IO verifiers
cannot be corrected in phase 2 when we do the initial scan of the
AGs. Correcting some errors cannot be done until a full rebuild of
the trees is done in phase 5.
Hence we can end up with a "clean" AGF/AGI buffer but have a
EFSCORRUPTED error on the buffer. This results in an assert failing:
and repair not beign able to fix the problems it has tripped over.
Hence the assert that we corrected all corruptions in the buffers
is not valid and should be removed.
Reported-by: Hans Kraus <hans.w.kraus@gmx.at> Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Fri, 11 Jul 2014 02:00:28 +0000 (12:00 +1000)]
libxfs: clear the buffer error while the buffer is locked
When releasing a buffer, the error shoul dbe cleared while the lock
is still held on the buffer to avoid racing with a new user of the
buffer.
This was pointed out in review of commit 6af7c1e ("libxfs: reused
invalidated buffers leak state and data") but the version committed
didn't have the fix. Thanks to Christoph Hellwig for checking and
pointing out the oversight.
Reported-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
We then capture this bad inode number later in the same function
either in the same pass or in a later phase and junk the entry.
When we junk the entry, we write a "/" over the first character of
the dirent name, which is then detected up later by the directory
rebuild and ignored.
The issue with this is that the directory buffer can be written to
disk between the dirent being marked with BADFSINO and the directory
rebuild processing in phase 6, resulting in the directory block
verifier firing this error:
Invalid inode number 0xfeffffffffffffff
xfs_dir_ino_validate: XFS_ERROR_REPORT
Metadata corruption detected at block 0x11fbb698/0x1000
libxfs_writebufr: write verifer failed on bno 0x11fbb698/0x1000
And so will not write the *corrupt block* to disk. The result is
that we don't repair a corruption in the directory block correctly
and subsequent repair runs continue to find problems with the
directory.
We really don't need both BADFSINO *and* overwriting the dirent name
with "/" to mark an entry as junked. They both mean exactly the same
thing, so get rid of BADFSINO and only use the name junking to mark
dirents as bad. This prevents the directory data block verifier from
triggering on bad inode numbers, and so the later reread of the
block will find the junked entries correctly.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Tue, 8 Jul 2014 00:36:39 +0000 (10:36 +1000)]
repair: fix quota inode handling in secondary superblocks
Changes to support separate project quota inodes changed the way
quota inodes got written to the superblock. The current code is
tailored for the needs to the kernel, where the inodes should only
be written if certain falgs are set saying a quota type is enabled.
Unfortunately, when recovering a corrupt secondary superblock, we
need to unconditionally write the quota inode fields after we
unconditionally zero the quota flags field. The result of this bug
is that the bad quota inode fields cannot be cleared and hence
always are reported by bad by repair in subsequent runs.
Fix this by directly clearing the quota inodes in the superblock
buffers so that we do need to set special flags to get
xfs_sb_to_disk() to do the right thing as setting flags leave bad
flag values in the superblock instead of bad inode numbers....
Also, when clearing the inode numbers, write them as NULLFSINO
rather than 0 as this is what the kernel will write them as if quota
is turned off.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Tue, 8 Jul 2014 00:36:39 +0000 (10:36 +1000)]
libxfs: reused invalidated buffers leak state and data
When rebuilding a bad directory, repair first truncates away all the
blocks in the directory, good or bad. This removes blocks from the
bmap btree, and when those blocks are freed the bmap btree code
invalidates them. This marks the buffers LIBXFS_B_STALE so that we
don't try to write stale data from that buffer at a later time.
However, when rebuilding the directory, blocks may get reallocated
and we reuse the underlying buffers. This has two problems.
The first is that if the buffer was previously detected as having a
verifier error (i.e. an error that is leading to the block being
freed and the buffer being invalidated) then the error might still
be held in b_error. Hence the libxfs code needs to ensure that
b_error does not leak from one buffer usage context to another
after invalidation.
The second problem is that when new data is written into a buffer,
it no longer has stale contents. Hence when we write the buffer, we
need to clear the LIBXFS_B_STALE flag to ensure that the new data
gets written.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Tue, 8 Jul 2014 00:36:34 +0000 (10:36 +1000)]
repair: handle directory block corruption in phase 6
This should only occur in no-modify mode, but when we fail to find
the last extent in a directory btree due to corruption we need to
trash the directory if it's the first data block we find the error
on. That is because there is nothing to recover from the directory,
and if we try to scan it xfs_reapir segv's because nothing has been
read from disk.
Also catch a memory allocation failure in this code, too.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Tue, 8 Jul 2014 00:31:36 +0000 (10:31 +1000)]
xfs_db: write command broken on 64 bit values
convert_args() has problesm with 64 bit fields because it tries to
shift them by 64 bits. The result of doing so is undefined by the C
standard, and so results in the unexpected behaviour of the result
being being the original value unchanged rather than 0. Hence you
can't write 64 bit fields because the code thinks that all values
other than 0 are out of range.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Tue, 8 Jul 2014 00:30:52 +0000 (10:30 +1000)]
repair: support more than 25 ACLs
v5 superblock supports many more than 25 ACLs on an inode, but
xfs_repair still thinks that the maximum is 25. This slipped through
becase the reapir code does not share any of the kernel side ACL
code in libxfs, and instead has all it's own internal ACL
definitions.
Fix the repair code to support more than 25 ACLs and update
the ACL definitions to match the kernel definitions. In doing so,
this tickles a off-by-one bug on remote attribute maximum sizes
that is already fixed in the kernel code. So in addition to fixing
the repair code, this patch pulls in parts of the following kernel
commits:
bba719b5 xfs: fix off-by-one error in xfs_attr3_rmt_verify 0a8aa193 xfs: increase number of ACL entries for V5 superblocks
Reported-by: Michael L. Semon <mlsemon35@gmail.com> Tested-by: Michael L. Semon <mlsemon35@gmail.com> Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Jan Kara [Tue, 8 Jul 2014 00:30:28 +0000 (10:30 +1000)]
repair: Repair directory block CRC mismatches
It can happen that just CRC doesn't match for directory blocks. In that
case xfs_repair will just report the error but won't fix anything (as
further checking of the block doesn't reveal any problems). Make sure we
recompute and write out new CRC when we failed verification during
reading.
Signed-off-by: Jan Kara <jack@suse.cz> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
This is because xfs_sb_from_disk doesn't fill in sb_crc,
but xfs_sb_to_disk(XFS_SB_ALL_BITS) does write the in-memory
CRC to disk - so we get uninitialized memory on disk.
Fix this by always initializing sb_crc to 0 when we read
the superblock, and masking out the CRC bit from ALL_BITS
when we write it.
This same fix has already been sent for kernelspace.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Thu, 19 Jun 2014 02:14:01 +0000 (12:14 +1000)]
xfsprogs: add mount options to xfs.5 manpage
This is a straight cut and paste from the util-linux
mount manpage to xfs.5.
It's pretty much impossible for util-linux to keep up
with every filesystem out there, and Karel has more than
once expressed a wish that mount options move into fs-specific
manpages.
So, here we go.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Thu, 19 Jun 2014 02:12:36 +0000 (12:12 +1000)]
xfs_fsr: ensure the line we read from leftofffile is null terminated
Ensure that the string we read from leftofffile is NULL
terminated; the buffer gets passed to strchr(), so
it's important that we ensure it ends with NULL.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Junxiao Bi [Thu, 19 Jun 2014 02:11:48 +0000 (12:11 +1000)]
xfs_copy: fix data corruption of target
The unit of XFS_AGFL_DADDR(mp) is "basic block" whose size is "BBSIZE"
(512 bytes), so when "source_sectorsize" is not 512, it will cause the
target a corrupted filesystem.
Signed-off-by: Junxiao Bi <junxiao.bi@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Thu, 19 Jun 2014 02:11:20 +0000 (12:11 +1000)]
mkfs.xfs: don't call blkid_get_topology on existing regular files
If we encounter a target that's really a regular file,
even without "-d file..." on the cmdline, call
platform_findsizes() instead of blkid_get_topology to
try to discover the "sector size" via the fsgeom() call.
Otherwise mkfs.xfs will try to do direct IO with a default
512 sector size, and if the underlying file has different
DIO requirements, mkfs will fail.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Thu, 19 Jun 2014 02:10:23 +0000 (12:10 +1000)]
xfsprogs: try to handle mkfs of a file on 4k sector device
Try the xfs geometry ioctl if the mkfs target resides
in a file; this gives us the equivalent of a device
sector size.
If this fails, and there's a sector size mismatch
between the host FS and the filesystem, then mkfs might
fail - but that's no worse than it's been before.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:07:36 +0000 (09:07 +1000)]
db: add finobt support to metadump
Include the free inode btree in metadump images. If the source fs
is finobt-enabled, run an additional scan_btree() of the finobt.
Since the private 'agi' scanfunc_ino() parameter is unused, change
the private parameter to a flag that indicates whether the current
scan is for the inobt or finobt. If the latter, we skip copying the
actual inode chunks as this work is already performed by the inobt
scan.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:07:11 +0000 (09:07 +1000)]
repair: reconstruct the finobt in phase 5
Support reconstruction of the finobt in phase 5 of xfs_repair. We
create a new cursor for the finobt and write the in-core records
that contain free inodes to the tree. Finally, pass the cursor
along to build_agi() to include the finobt root and level count in
the agi header.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:07:06 +0000 (09:07 +1000)]
repair: helpers for finding in-core inode records w/ free inodes
Add the findfirst_free_inode_rec() and next_free_ino_rec() helpers
to assist scanning the in-core inode records for records with at
least one free inode. These will be used to determine what records
are included in the free inode btree.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:07:00 +0000 (09:07 +1000)]
repair: pull the build_agi() call up out of the inode tree build
Pull the build_agi() call out of build_ino_tree() in phase 5. This
is to prepare for finobt support, in which build_agi() will require
context from multiple inode tree reconstructions (both the inode
allocation tree and free inode tree, when it exists).
Create the new 'agi_stat' structure to carry the requisite state
from the build_ino_tree() operation to build_agi().
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:06:54 +0000 (09:06 +1000)]
repair: pass btree block magic as param to build_ino_tree()
A minor cleanup to build_ino_tree() to provide the appropriate
magic value for btree block initialization from the caller. This
facilitates use of separate magic values for finobt blocks when
building the free inode btree.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:06:47 +0000 (09:06 +1000)]
repair: phase 2 finobt scan
If one exists, scan the free inode btree in phase 2 of xfs_repair.
We use the same general infrastructure as for the inobt scan, but
trigger finobt chunk scan logic in in scan_inobt() via the magic
value.
The new scan_single_finobt_chunk() function is similar to the inobt
equivalent with some finobt specific logic. We can expect that
underlying inode chunk blocks are already marked used due to the
previous inobt scan. We can also expect to find every record
tracked by the finobt already accounted for in the in-core tree
with equivalent (and internally consistent) inobt record data.
Spit out a warning on any divergences from the above and add the
inodes referenced by the current finobt record to the appropriate
in-core tree.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Mon, 26 May 2014 23:06:17 +0000 (09:06 +1000)]
mkfs: finobt mkfs support
Add the 'finobt' metadata option to mkfs to format an fs with free
inode btree support. If enabled, initialize the associated AGI
header fields and btree root block.
Also, do the initialization of the superblock version and feature
bits (including the new finobt flag) a bit earlier. These fields
must now be initialized prior to the use of XFS_PREALLOC_BLOCKS(),
as the latter returns a value that depends on whether a finobt root
btree block is reserved.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:57:45 +0000 (07:57 +1000)]
xfs: enable the finobt feature on v5 superblocks
Add the finobt feature bit to the list of known features. As of
this point, the kernel code knows how to mount and manage both
finobt and non-finobt formatted filesystems.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:57:24 +0000 (07:57 +1000)]
xfs: update the finobt on inode free
An inode free operation can have several effects on the finobt. If
all inodes have been freed and the chunk deallocated, we remove the
finobt record. If the inode chunk was previously full, we must
insert a new record based on the existing inobt record. Otherwise,
we modify the record in place.
Create the xfs_ifree_finobt() function to identify the potential
scenarios and update the finobt appropriately.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:57:15 +0000 (07:57 +1000)]
xfs: refactor xfs_difree() inobt bits into xfs_difree_inobt() helper
Refactor xfs_difree() in preparation for the finobt. xfs_difree()
performs the validity checks against the ag and reads the agi
header. The work of physically updating the inode allocation btree
is pushed down into the new xfs_difree_inobt() helper.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:54:04 +0000 (07:54 +1000)]
xfs: use and update the finobt on inode allocation
Replace xfs_dialloc_ag() with an implementation that looks for a
record in the finobt. The finobt only tracks records with at least
one free inode. This eliminates the need for the intra-ag scan in
the original algorithm. Once the inode is allocated, update the
finobt appropriately (possibly removing the record) as well as the
inobt.
Move the original xfs_dialloc_ag() algorithm to
xfs_dialloc_ag_slow() and fall back as such if finobt support is
not enabled.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:53:36 +0000 (07:53 +1000)]
xfs: insert newly allocated inode chunks into the finobt
A newly allocated inode chunk, by definition, has at least one
free inode, so a record is always inserted into the finobt.
Create the xfs_inobt_insert() helper from existing code to insert
a record in an inobt based on the provided BTNUM. Update
xfs_ialloc_ag_alloc() to invoke the helper for the existing
XFS_BTNUM_INO tree and XFS_BTNUM_FINO tree, if enabled.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:53:18 +0000 (07:53 +1000)]
xfs: update inode allocation/free transaction reservations for finobt
Create the xfs_calc_finobt_res() helper to calculate the finobt log
reservation for inode allocation and free. Update
XFS_IALLOC_SPACE_RES() to reserve blocks for the additional finobt
insertion on inode allocation. Create XFS_IFREE_SPACE_RES() to
reserve blocks for the potential finobt record insertion on inode
free (i.e., if an inode chunk was previously fully allocated).
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:53:07 +0000 (07:53 +1000)]
xfs: support the XFS_BTNUM_FINOBT free inode btree type
Define the AGI fields for the finobt root/level and add magic
numbers. Update the btree code to add support for the new
XFS_BTNUM_FINOBT inode btree.
The finobt root block is reserved immediately following the inobt
root block in the AG. Update XFS_PREALLOC_BLOCKS() to determine the
starting AG data block based on whether finobt support is enabled.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:52:47 +0000 (07:52 +1000)]
xfs: reserve v5 superblock read-only compat. feature bit for finobt
Reserve a v5 read-only compatibility feature bit for the finobt and
create the xfs_sb_version_hasfinobt() helper to determine whether
an fs has the feature enabled.
The finobt does not change existing on-disk structures, but must
remain consistent with the ialloc btree. Modifications from older
kernels would violate that constrant. Therefore, we restrict older
kernels to read-only mounts of finobt-enabled filesystems.
Note that this does not yet enable the ability to rw mount a finobt
fs (by setting the feature bit in the XFS_SB_FEAT_RO_COMPAT_ALL
mask).
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Brian Foster [Tue, 20 May 2014 21:52:26 +0000 (07:52 +1000)]
xfs: refactor xfs_ialloc_btree.c to support multiple inobt numbers
The introduction of the free inode btree (finobt) requires that
xfs_ialloc_btree.c handle multiple trees. Refactor xfs_ialloc_btree.c
so the caller specifies the btree type on cursor initialization to
prepare for addition of the finobt.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Roger Willcocks [Tue, 20 May 2014 08:30:44 +0000 (18:30 +1000)]
libxfs: fix compile error when libxfs header used in C++ code
xfs_ialloc.h:102: error: expected ',' or '...' before 'delete'
Simple parameter rename, no changes to behaviour.
Signed-off-by: Roger Willcocks <roger@filmlight.ltd.uk> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Junxiao Bi [Tue, 20 May 2014 08:30:23 +0000 (18:30 +1000)]
xfs_copy: use exit() to replace killall()
Sending a SIGKILL signal to child thread will terminate the whole process,
xfs_copy will return an error value 137. This cause confuse for script to
know whether the copy successes.
Calling exit() in main thread can terminate the whole process and return the
right value. Replace killall()+abort() with exit(1) to match the old way
exit in error case. Also remove killall()+pthread_exit(NULL) since return 0
will be followed by an exit(0) to terminate the process.
[ Christoph Hellwig:
Btw, I think the reason for this cruft is that xfs_copy was originally
written using the IRIX sproc interface, and the port to pthreads didn't
remove this gem:
Mark Tinguely [Tue, 20 May 2014 08:30:11 +0000 (18:30 +1000)]
libxfs: dont free xfs_inode until complete
Originally, the xfs_inode are released upon the first
call to xfs_trans_cancel, xfs_trans_commit, or
inode_item_done. This code used the log item lock field
to prevent the release of the inode on the next call to
one of the above functions. This is a unusual use of the
log item lock field which is suppose to specify which lock
is to be release on transaction commit or cancel. User
space does not perform locking in transactions..
Unfortunately, this breaks any code that relies on multiple
transaction operations. For example, adding an extended
attribute to an inode that does not have an attribute fork
will fail:
This patch does the following:
1) Removes the iput from the transaction completion and
requires that the xfs_inode allocators call IRELE()
when they are done with the pointer. The real time
inodes are pointed to by the xfs_mount and have a longer
lifetime.
2) Removes libxfs_trans_iput() because transaction entries
are removed in transaction commit and cancel.
3) Removes libxfs_trans_ihold() which is an obsolete interface.
4) Removes the now unneeded ili_ilock_flags from the
xfs_inode_log_item structure.
Signed-off-by: Mark Tinguely <tinguely@sgi.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Fri, 9 May 2014 04:50:28 +0000 (14:50 +1000)]
repair: don't grind CPUs with large extent lists
When repairing a large filesystem with fragemented files, xfs_repair
can grind to a halt burning multiple CPUs in blkmap_set_ext().
blkmap_set_ext() inserts extents into the blockmap for the inode
fork and it keeps them in order, even if the inserts are not done in
order.
The ordered insert is highly inefficient - it starts at the first
extent, and simple walks the array to find the insertion point. i.e.
it is an O(n) operation. When we have a fragemented file with a
large number of extents, the cost of the entire mapping operation
is rather costly.
The thing is, we are doing the insertion from an *ordered btree
scan* which is inserting the extents in ascending offset order.
IOWs, we are always inserting the extent at the end of the array
after searching the entire array. i.e. the mapping operation cost is
O(N^2).
Fix this simply by reversing the order of the insert slot search.
Start at the end of the blockmap array when we do almost all
insertions, which brings the overhead of each insertion down to O(1)
complexity. This, in turn, results in the overall map building
operation being reduced to an O(N) operation, and so performance
degrades linearly with increasing extent counts rather than
exponentially.
While there, I noticed that the growing of the blkmap array was only
done 4 extents at a time. When we are dealing with files that may
have hundreds of thousands of extents, growing th map only 4 extents
at a time requires excessive amounts of reallocation. Reduce the
reallocation rate by increasing the grow increment according to how
large the array currently is.
The result is that the test filesystem (27TB, 30M inodes, at ENOSPC)
takes 5m10s to *fully repair* on my test system, rather that getting
15 (of 60) AGs into phase three and sitting there burning 3-4 CPUs
making no progress for over half an hour.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Fri, 9 May 2014 04:50:12 +0000 (14:50 +1000)]
repair: don't unlock prefetch tree to read discontig buffers
The way discontiguous buffers are currently handled in prefetch is
by unlocking the prefetch tree and reading them one at a time in
pf_read_discontig(), inside the normal loop of searching for buffers
to read in a more optimized fashion.
But by unlocking the tree, we allow other threads to come in and
find buffers which we've already stashed locally on our bplist[].
If 2 threads think they own the same set of buffers, they may both
try to delete them from the prefetch btree, and the second one to
arrive will not find it, resulting in:
fatal error -- prefetch corruption
To fix this, simply abort the buffer gathering loop when we come
across a discontiguous buffer, process the gathered list as per
normal, and then after running the large optimised read, check to
see if the last buffer on the list is a discontiguous buffer.
If is is discontiguous, then issue the discontiguous buffer read
while the locks are not held. We only ever have one discontiguous
buffer per read loop, so it is safe just to check the last buffer in
the list.
The fix is loosely based on a a patch provided by Eric Sandeen, who
did all the hard work of finding the bug and demonstrating how to
fix it.
Reported-by:Eric Sandeen <sandeen@redhat.com> Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Nathan Scott [Sat, 3 May 2014 06:10:38 +0000 (16:10 +1000)]
Fix msgfmt warning when building the German translation
Use the same header present in the Polish language files to
resolve the following warning in the de.po build:
[MSGFMT] de.mo
de.po:7: warning: header field 'Language' missing in header
Nathan Scott [Sat, 3 May 2014 06:09:03 +0000 (16:09 +1000)]
Fix 32 bit build warning in libxfs, xfs_daddr_t printing
Add the usual type casts to resolve the following warnings:
rdwr.c: In function 'libxfs_getbufr_map':
rdwr.c:499:4: warning: format '%lx' expects argument of type 'long unsigned int', but argument 5 has type 'xfs_daddr_t' [-Wformat]
rdwr.c:499:4: warning: format '%lx' expects argument of type 'long unsigned int', but argument 6 has type 'xfs_daddr_t' [-Wformat]
Dave Chinner [Thu, 1 May 2014 23:31:33 +0000 (09:31 +1000)]
repair: detect and handle attribute tree CRC errors
Currently the attribute code will not detect and correct errors in
the attribute tree. It also fails to validate the CRCs and headers
on remote attribute blocks. Ensure that all the attribute blocks are
CRC checked and that the processing functions understand the correct
block formats for decoding.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:31:28 +0000 (09:31 +1000)]
repair: handle remote symlink CRC errors
We can't really repair broken symlink buffer contents, but we can at
least warn about it and correct the CRC error so the symlink is
again readable.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:31:17 +0000 (09:31 +1000)]
repair: report AG btree verifier errors
When we scan the filesystem freespace and inode maps in phase 2, we
don't report CRC errors that are found. We don't really care from a
repair perspective, because the trees are completely rebuilt from
the ground up in phase 5, but froma checking perspective we need to
inform the user that we found inconsistencies.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:31:11 +0000 (09:31 +1000)]
repair: detect CRC errors in AG headers
repair doesn't currently detect verifier errors in AG header
blocks - apart from the primary superblock they are not detected.
They are, fortunately, corrected in the important cases (AGF, AGI
and AGFL) because these structures are rebuilt in phase 5, but if
you run xfs_repair in checking mode it won't report them as bad.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:31:04 +0000 (09:31 +1000)]
repair: detect and correct CRC errors in directory blocks
repair doesn't currently verifier errors in directory blocks - they
cause repair to ignore blocks and hence fail because it can't read
critical blocks from the directory.
Fix this by having the directory buffer read code detect a verifier
error and retry the read without the verifier if the verifier has
detected an error. Then pass the verifer error with the successfully
read buffer back to the caller, so the caller can handle the error
appropriately. In most cases, this is simply marking the directory
as needing a rebuild, so once the directory entries have been
checked and repaired, it will rewrite all the directory buffers
(including the clean ones) and in the process recalculate all the
the CRC on the directory blocks.
Hence pure CRC errors in directory blocks are now handled correctly
by xfs_repair.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:30:57 +0000 (09:30 +1000)]
repair: ensure prefetched buffers have CRCs validated
Prefetch currently does not do CRC validation when the IO completes
due to the optimisation it performs and the fact that it does not
know what the type of metadata into the buffer is supposed to be.
Hence, mark all prefetched buffers as "suspect" so that when the
end user tries to read it with a supplied validation function the
validation is run even though the buffer was already in the cache.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:30:47 +0000 (09:30 +1000)]
db: verify buffer on type change
Currently when the type command is run, we simply change the type
associated with the buffer, but don't verify it. This results in
unchecked CRCs being displayed. Hence when changing the type, run
the verifier associated with the type to determine if the buffer
contents is valid or not.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Dave Chinner [Thu, 1 May 2014 23:30:39 +0000 (09:30 +1000)]
db: don't claim unchecked CRCs are correct
Currently xfs_db will claim the CRC on a structure is correct if the
buffer is not marked with an error. However, buffers may have been
read without a verifier, and hence have not had their CRCs
validated. in this case, we shoul dreport "unchecked" rather than
"correct". For example:
xfs_db> fsb 0x6003f
xfs_db> type dir3
xfs_db> p
dhdr.hdr.magic = 0x58444433
dhdr.hdr.crc = 0x2d0f9c9d (unchecked)
....
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
Mark Tinguely [Mon, 14 Apr 2014 06:15:05 +0000 (16:15 +1000)]
logprint: handle split EFI entry
xfs_logprint does not correctly handle EFI entries that
are split across two log buffers. xfs_efi_copy_format()
falsely interrupts the truncated size of the split entry
as being a corrupt entry.
If the first log entry has enough information, namely the
number of extents in the entry and the identifier, then
display this information and a warning that this entry is
truncated. Otherwise, if there is not enough information in
the first log buffer, then print a message that the EFI decode
was not possible. These messages are similar to split inode
entries.
Example of a continued entry:
Oper (336): tid: f214bdb len: 44 clientid: TRANS flags: CONTINUE
EFI: #regs: 1 num_extents: 2 id: 0xffff880804f63900
EFI free extent data skipped (CONTINUE set, no space)
Reported-by: Michael L. Semon <mlsemon35@gmail.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 14 Apr 2014 06:13:58 +0000 (16:13 +1000)]
libxfs: remove never-read "offset" assignment in readbufr_map & writebufr
libxfs_readbufr_map() & libxfs_writebufr() iterate
over bp->b_map[] and read each chunk. The loops start
out correctly, getting the offset from bm_bn and the
length from bm_len. After the IO it correctly
advances the target buffer pointer by len, but then
inexplicably advances "offset" by len as well. The
whole point of this exercise is to handle discontiguous
ranges - marching offset along by length of IO done
is incorrect.
Thankfully offset is immediately reset to the proper
value again at the top of the loop for the next range,
so this is harmless, other than being confusing.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 14 Apr 2014 06:13:54 +0000 (16:13 +1000)]
xfs_io: fix random pread/pwrite to honor offset
xfs_io's pread & pwrite claim to support a random IO mode
where it will do random IOs between offset & offset+len.
However, offset was ignored, and we did the IOs between 0
and len instead.
Clang caught this by pointing out that the calculated/normalized
"offset" variable was never read.
(NB: If the range is larger than RAND_MAX, these functions don't
work, but that's always been true, so I'll leave it for another
day...)
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 14 Apr 2014 06:13:50 +0000 (16:13 +1000)]
xfsprogs: remove write-only assignments
There are many instances where variable assignments are made,
but never read (or are re-assigned before they are read).
The Clang static analyzer finds these.
Here's a chunk of what I think are trivial removals of such
assignments; other detections point to more serious problems
(or are shared w/ kernel code so should probably be fixed there
first).
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
Eric Sandeen [Mon, 14 Apr 2014 06:13:44 +0000 (16:13 +1000)]
mkfs: catch unknown format in protofile parsing
As the code stands today we can't get an unknown format in the
last case statement, but Coverity warns that if we ever do, we'll
use an uninitialized "ip" in the call to libxfs_trans_log_inode().
Adding a default: case to catch unknown formats is defensive and
makes the checker happy.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
projects / thirdparty / xfsprogs-dev.git / log
