Michael L. Semon [Fri, 14 Jun 2013 07:00:34 +0000 (03:00 -0400)]
xfsprogs: define umode_t for build if not defined already
umode_t has not been exported to the kernel private headers since
around kernel 3.2.0-rc7. Add a check for umode_t and define it
if it has not been defined already.
Signed-off-by: Michael L. Semon <mlsemon35@gmail.com> Reviewed-by: Ben Myers <bpm@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com>
Dave Chinner [Thu, 9 May 2013 16:20:13 +0000 (11:20 -0500)]
xfs_logprint: fix continuation transactions
As demonstrated by xfs/295, continuation transactions cause of
problems for xfs_logprint. The failure demonstrated by the test is
that the buffer log format structures are variable sized on disk -
the dirty bitmap is sized according to the buffer length, not fixed
to the length of the maximum supported buffer size.
xfs_logprint assumes that the buf log format reocrds are of fixed
size, and so when a short buffer is found it fails to handle it
properly and treats it like a continuation record. This causses the
opheader pointer to be incremented incorrectly and then logprint
wanders off into a dark corner and gets eaten by a grue.
While fixing this, make the xlog_print_record code that does the
transaction opheader walking a little easier to read and stop it
from outputting binary data direct to the console by converting the
no-data-print case to use a hex dumping loop.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Dave Chinner [Thu, 9 May 2013 13:19:06 +0000 (08:19 -0500)]
xfsprogs: updata libxlog to current kernel code
Update the libxlog log recovery code to match the current 3.8-rc2
kernel code and ensure all the callers work correctly.
Note: while this introduces CRC validation infrastructure, it is
currently short-circuited as it is not clear what to do with log
buffers that fail CRC checking. We're only reading the log to
determine it is clean/dirty or dumping the contents for analysis, so
it's not clear what to do with CRC validation errors yet, or even
if there is any commonality with the kernel handling. This will need
to be revisited as the situation clarifies.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Dave Chinner [Thu, 9 May 2013 13:18:51 +0000 (08:18 -0500)]
xfsprogs: add CRC32c infrastructure
Pull the generic crc32(c) code from the kernel and add it to libxfs.
Modify it to build in the libxfs environment, and drop the bigendian
CRC version as it is unused by XFS, which uses the little endian
version so that it can be hardware accelerated using native
instructions on x86-64 CPUs.
Also wire up the self-test code in the crc32 module to the build
infrastructure and make passing the self test a build dependency.
This prevents xfsprogs from being built on platforms that the CRC
algorithm does not work on and hence ensures the tools do not write
bad CRCs to disk as a result of a broken calculation.
Also pull the XFS CRC helper functions across in preparation for
using the CRC functions in libxfs.
XXX: something in the CRC table generation breaks the debian package
build. It fails to build libxfs as a dependency of mkfs.xfs. Works
fine outside the debian build environment, so I'm not sure what the
issue is yet. Most likely it is the execution path of the
gen_crc32table binary that is built...
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Dave Chinner [Thu, 9 May 2013 12:22:15 +0000 (07:22 -0500)]
xfsprogs: Die dir1 Die!
Version 1 directories have never been supported on linux, and we base the
default on pre-Irix 6.2 kernels (~1998). A recent xfs_repair debugging session
on #xfs determined that dir v1 support in xfs_repair has been broken since May
26, 2008 when the ascii case insensitivity support was added to userspace.
Seeing that the code has been broken for roughly 5 years and the first time that
it was noticed was a couple of days ago, it is clearly rarely required, rarely
used and completely untested.
Following the time-honoured X server deprecation model, if it's been broken for
several years and nobody has noticed, then it can and should be removed. So,
rather than trying to fix something we can't test and very, very few people care
about, let's just remove it.
For xfs_repair, some of the checking code is shared with the attribute repair
code. Once all the dirv1 specific code is removed, there isn't a whole lot left,
so move it to attr_repair.c and we can kill the dir.[ch] files completely.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Dave Chinner [Thu, 9 May 2013 12:16:09 +0000 (07:16 -0500)]
xfs_fsr: file reads should be O_DIRECT
When running xfs_fsr on a sparse filesystem image containing
approximately 8 million extents and 80GB of data, I noticed that the
page cache grew and consumed all the memory in the machine. It turns
out that xfs_fsr is using direct IO to write data, but buffered IO
to read data. Convert the read side to use direct IO to prevent page
cache blowouts.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Dave Chinner [Thu, 9 May 2013 12:11:50 +0000 (07:11 -0500)]
xfs_logprint: print all AGI unlinked buckets
When printing buffer contents, the AGI unlinked buckets are not
printed in transactional output. In normal dump format, they are
printed, but that format is generally not useful for log recovery
analysis. Add the same code to the transactional buffer dump.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Dave Chinner [Thu, 2 May 2013 12:59:20 +0000 (07:59 -0500)]
xfs_repair: validate on-disk extent count better
When scanning a btree format inode, we trust the extent count to be
in range. However, values of the range 2^31 <= cnt < 2^32 are
invalid and can cause problems with signed range checks. This
results in assert failures which validating the extent count such
as:
Eric Sandeen [Thu, 25 Apr 2013 15:03:47 +0000 (15:03 +0000)]
xfsprogs: Fix manpages for missing or incorrect options
Add valid options which aren't in manpages, and
remove invalid options which are in manpages:
* Document -V (show version and exit) for many manpages.
* Remove -? option from xfs_estimate.8
* Document -p passes, -d (debug) and -g (syslog) in xfs_fsr.8
* Document -n (O_NONBLOCK) in xfs_io.8
* Document -v (print overwrite) in xfs_logprint.8
* Document -m max_extents in xfs_metadump.8
* Document -p (preallocate) in xfs_mkfile.8
Brian Foster [Tue, 19 Mar 2013 13:23:35 +0000 (13:23 +0000)]
xfsprogs: reduce bb_numrecs in bno/cnt btrees when log consumes all agf space
The mkfs code currently creates a single free space extent record
for each of the bno and cnt btrees in each AG. The start block of
the record is pushed forward on the AG that hosts an internal log.
If the log happens to consume all available space in the AG, the
start block becomes equal to sb->sb_agblocks and thus invalid.
This causes xfs_repair to complain.
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- scan filesystem freespace and inode maps...
invalid start block 4096 in record 0 of bno btree block 1600/1
invalid start block 4096 in record 0 of cnt btree block 1600/2
- found root inode chunk
...
xfs_repair appears to correct the numrecs value such that subsequent
checks are successful. The sequence above is pulled from xfstests
test #250, which fails due to this behavior.
Modify mkfs.xfs such that we check the block count value of the
free space record for the log AG after the log is accounted for. If
no space is left for the record, reset the record count to 0.
Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Eric Sandeen [Sat, 9 Mar 2013 15:21:55 +0000 (15:21 +0000)]
xfsprogs: skip freelist scans of corrupt agf
If an agf has bad values in the freelist, this can wreak
havoc if, for example, first > last and the loop
never exits; we index agfl->agfl_bno[i] off into the weeds.
If they're off, warn about it and skip the scan.
This is done both in xfs_check and xfs_db's freespace cmd.
Also fix uninit'd variable "i" from previous, similar fix
for xfs_repair.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Eric Sandeen [Sat, 2 Mar 2013 21:23:12 +0000 (21:23 +0000)]
xfsprogs: xfs_repair skip freelist scan of corrupt agf in no-modify mode
In xfs_repair's no-modify mode (-n), verify_set_agf doesn't fix up
bad freelist blocks that it finds. When we get to scan_freelist,
this can wreak havoc if, for example, first > last and the loop
never exits; we index agfl->agfl_bno[i] off into the weeds.
To fix this, re-check the values in no-modify mode, and if
they're off, warn about it and skip the scan.
Reported-by: Ole Tange <tange@binf.ku.dk> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Rich Johnston <rjohnston@sgi.com> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
Eric Sandeen [Sat, 26 Jan 2013 22:40:31 +0000 (22:40 +0000)]
xfs_fsr: fix attribute no_change_count logic
As it stands today, if no_change_count++ isn't > 10,
we will reset it to 0. There's no way to get above 1
(let alone 10) so this isn't working as intended.
If we see progress (last_forkoff != tbstat.bs_forkoff)
*then* we sould reset the no_change_count counter to 0.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Eric Sandeen [Sat, 26 Jan 2013 22:40:27 +0000 (22:40 +0000)]
libxfs: fix setup_cursor array allocation
setup_cursor() wants an array of xfs_agbno_t's, but
it allocated a multiple of *pointers* to xfs_agbno_t's.
xfs_agbno_t is 4 bytes, so this is harmless other than
allocating twice as much memory as needed on a 64-bit
machine.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Eric Sandeen [Fri, 25 Jan 2013 21:10:22 +0000 (21:10 +0000)]
xfsprogs: Fix possible unallocated memory access in fiemap
(Based on original patch by Lukas Czerner & comments by Dave Chinner)
Currently we could access unallocated memory in fiemap because we're
using uninitialized variable 'fiemap' in fiemap_f(). In fact this has
been spotted on x390s machine where xfs_io would segfault.
The problem happens in the for cycle which seems to be intended to
compute the header item spacing. However at that point the fiemap
structure has just been allocated and does not contain any extents
yet, so it is entirely useless and it never actually worked.
This patch delays the format calculation until the first batch
of extents has come in for analysis.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Jeff Liu [Tue, 19 Feb 2013 05:31:25 +0000 (05:31 +0000)]
xfsprogs: sync the changes in transaction log space reservations to user space
Sync the kernel code changes regarding transaction log space
reservations to user space.
As we have splitted the calculation of attrset log space reservations
into mount time and runtime in kernel code, here we need to fix
max_attrset_trans_res_adjust() to reflect this change.
Signed-off-by: Jie Liu <jeff.liu@oracle.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Ben Myers [Thu, 14 Feb 2013 16:54:09 +0000 (10:54 -0600)]
xfsprogs: update 'make deb' to use tarball
This patch changes the build process so that 'make deb' uses the same
process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus
This is needed so that you can create a tarball in a bare release
tree, when .git is not available.
* Modify the SRCTAR target to include files from .gitcensus and use tar
instead of git archive.
* Modify the SRCTARINC files to include .gitcensus, and include
.gitcensus in the 'make realclean' target.
* remove the 'make source-link' target.
Signed-off-by: Ben Myers <bpm@sgi.com> Reviewed-by: Nathan Scott <nathans@debian.org> Reviewed-by: Dave Chinner <dchinner@redhat.com>
Andrew Dahl [Mon, 14 Jan 2013 18:16:02 +0000 (12:16 -0600)]
xfsprogs: Refactor release scripts to conform to using git archive
Refactored release scripts to conform to using git archive
When generating a release, there is a risk of missing necessary
source files. This is fixed by using git archive, which also
fixes the lack of conformity between the xfs utilities. As well,
some files may be stale during packaging. This is fixed with a
clean at the beginning of release generation.
Signed-off-by: Andrew Dahl <adahl@sgi.com> Reviewed-by: Ben Myers <bpm@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com>
Eric Sandeen [Wed, 2 Jan 2013 23:03:52 +0000 (17:03 -0600)]
xfs_logprint: Handle continued inode transactions
xlog_print_trans_inode() has a special case for 2
specific op_head->oh_len lengths. If it matches
sizeof(xfs_inode_log_format_32_t) or
sizeof(xfs_inode_log_format_64_t), it assumes that
it's got an inode, and attempts to convert it and
print it accordingly.
However, if we arrive here via an op header which
is continued, then the length is simply a continuation
of the previous op, and it might *randomly* match the
size of one of the inode log formats, and thus get parsed
incorrectly.
Change the caller to pass in whether or not it's a continued
op, so that it can be handled correctly.
Tested by running xfs_logprint of TEST_DEV in xfsprogs
after sequential tests; without this change it gets off
in the weeds eventually; with this fix, it lasts longer,
until it hits some other yet-unfixed logprint bug...
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Ben Myers <bpm@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com>
Eric Sandeen [Wed, 2 Jan 2013 23:02:17 +0000 (17:02 -0600)]
xfs_logprint: Handle multiply-logged inode fields
As xlog_print_trans_inode() stands today, it will error
out if more than one flag is set on f->ilf_fields:
xlog_print_trans_inode: illegal inode type
but this is a perfectly valid case, to have i.e. a data and
an attr flag set.
Following is a pretty big reworking of the function to
handle more than one field type set, mostly following
xlog_recover_inode_pass2() for logic.
I've tested this by a simple test such as creating one
file on an selinux box, so that data+attr is set, and
logprinting; I've also tested by running logprint after
subsequent xfstest runs (although we hit other bugs that
way).
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Ben Myers <bpm@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com>
blkid_get_topology() ignores devices which report 512
as their minimum & optimal IO size, but we should ignore
anything up to the physical sector size; otherwise hard-4k
sector devices will report a "stripe size" of 4k, and warn
if anything larger is specified:
# modprobe scsi_debug physblk_exp=3 num_parts=2 dev_size_mb=128
# mdadm --create /dev/md1 --level=0 --raid-devices=2 -c 4 /dev/sdb1 /dev/sdb2
# mkfs.xfs -f -d su=16k,sw=2 /dev/md1
mkfs.xfs: Specified data stripe unit 32 is not the same as the volume stripe unit 8
mkfs.xfs: Specified data stripe width 64 is not the same as the volume stripe width 16
...
but a stripe unit of 4k is pretty nonsensical. And that's even chosen by
default in this case, which is maybe even worse?
Eric Sandeen [Tue, 10 Apr 2012 04:34:12 +0000 (23:34 -0500)]
metadump: obfuscate symlinks by path component
xfs_metadump currently obfuscates entire symlinks without regard
to path components; this can lead to a corrupt image when restoring
a metadump containing extremely long symlinks:
Phase 3 - for each AG...
- scan and clear agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
component of symlink in inode 145 too long
problem with symbolic link in inode 145
cleared inode 145
... <more trail of woe>
Fix this by consolidating symlink obfuscation into a new
function which obfuscates one path component at a time.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Alex Elder <elder@kernel.org> Signed-off-by: Ben Myers <bpm@sgi.com>
Eric Sandeen [Sat, 8 Dec 2012 21:03:10 +0000 (21:03 +0000)]
xfsprogs: remove setfl from xfs_io
Doesn't seem to have worked for ages, and is (therefore)
apparently not ever used:
xfs_io> setfl
xfs_io> help setfl
setfl [-adx] -- set/clear append/direct flags on the open file
xfs_io> setfl -a
bad argument count 1 to setfl, expected 0 arguments
xfs_io> setfl -d
bad argument count 1 to setfl, expected 0 arguments
xfs_io> setfl
xfs_io>
At best, it seems intended to toggle the flag state, but
gives no feedback about current state. -x is in help but
not implemented, etc.
Just remove it.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Eric Sandeen [Sat, 8 Dec 2012 20:55:18 +0000 (20:55 +0000)]
xfsprogs: document all commands in xfs_io
Add missing command documentation to xfs_io(8) manpage.
fiemap, fpunch, chproj, lsproj, and setfl are all missing.
setfl seems to not work today in any case, and nothing
in xfstests uses it; I will send another patch to simply
remove it from xfs_io, as I don't think it's terribly useful,
and hasn't worked forever anyway.
Also fix references to the fallocate manpage, which is (now?)
in section 2, not section 3 of the man pages. (Since it's
a syscall, not a library function).
Eric Sandeen [Thu, 6 Dec 2012 21:52:54 +0000 (15:52 -0600)]
mkfs.xfs: go into multidisk mode when geometry is on cmdline
In the course of some other investigations, I found that
calc_default_ag_geometry() doesn't go into "multidisk" mode
unless stripe geometry is *detected* (i.e. by the blkid routines).
Specifying a geometry on the cmdline is *not* sufficient, because
we test (ft.dsunit | ft.dswidth) which are not set by the cmdline
options.
If we move the AG calculations to after we have set dsunit & dswdith,
then we'll pick up either cmdline-specified or blkid-detected
geometry, and go into "multidisk" mode for AG size/count
calculations in both cases.
So now for a ~5T fs, for example, we'd make several more
AGs:
Dave Chinner [Fri, 16 Nov 2012 01:14:48 +0000 (01:14 +0000)]
xfs_quota: correctly initialise the default path
When we initial xfs_quota, we place lots of information into the
fs_table. This includes all the devices/mount points the user has
specified as a global command line parameter to report on, as well
as all the paths under project quota control.
There is a "current path" pointer (fs_path) maintained by the code
that points somewhere into the fs_table. After the table is
initialised, fs_path always points to the last entry in the table,
and hence has to be re-initialised to point at the desired entry
before it can be used properly.
In the case of xfs_quota, if the command passed on the command line
is a non-global command, the command is called multiple times, each
time after the libxcmd args_command() callback is run. That starts
with an index of 0, and until the callback returns zero it will keep
passing whatever the last returned value was into the callback.
xfs_quota supplies such a callback, and it's purpose is to iterate
over the fs_table setting fs_path to the next mount point in the
table. IOWs, non-global quota functions get called once for each
mount point specified on the command line. However, it also means
that for global functions, the fs_path pointer is not
re-initialised and hence if there are project quotas configured the
fs_path pointer does not point to a mount point andhence commands
may malfunction..
The problem that demonstrated this is the report function. It does
it's own fs_table iteration if the command requires it, and so only
should be called once to avoid outputting the same information
multiple times. That's what the previous patch fixed by making the
command global, but this now has the effect of making commands that
need to operate on the device specified on the global command rely
on the fs_path variable pointing at that device.
Further, commands executed by the interactive method are always
treated as global commands, so the report command never worked as a
global command in the presence of a configured project quota setup.
Fix the problem by initialising the fs_path pointer correctly.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Dave Chinner [Fri, 9 Nov 2012 07:02:58 +0000 (07:02 +0000)]
xfs_quota: fix report command parsing
The report command line needs to be parsed as a whole not as
individual elements - report_f() is set up to do this correctly.
When treated as non-global command line, the report function is
called once for each command line arg, resulting in reports being
issued multiple times.
Set the command to be a global command so that it is only called
once.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Dave Chinner [Fri, 9 Nov 2012 07:02:57 +0000 (07:02 +0000)]
xfs_db: flush devices before exiting
Test 287 uses xfs_db to change 32-bit project ID support while the
filesystem is unmounted. On a large filesystem the test was failing
due to the mount not seeing the feature bit in the superblock.
xfs_db uses a different address space to the filesystem when it is
mounte dby the kernel, so the only way to keep them coherent is to
ensure that all buffered data is written to disk before the other
entity tries to read it. xfs_db uses buffered IO, but does not close
the devices when it exits, thereby leaving changes it has written in
the block device cache rather than on disk. Hence when th ekernel
tries to mount the filesystem, it reads what is on disk and does not
see xfs_db's changes.
Fix this by ensuring that xfs_db flushes it's changes to disk before
it exits by caling libxfs_device_close(). This fsyncs the data and
flushes the caches to ensure that it is present on disk before
xfs_db exits.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Mike Frysinger [Mon, 24 Sep 2012 23:39:38 +0000 (19:39 -0400)]
xfsprogs: install shared libs with +x bits
These are shared libs w/executable code, so make sure they have +x bits
set on them. Some kernels will proactively disallow executable mmaps if
the files lack +x bits. It's also the right thing to do.
Signed-off-by: Mike Frysinger <vapier@gentoo.org> Reviewed-by: Ben Myers <bpm@sgi.com> Signed-off-by: Ben Myers <bpm@sgi.com>
Eric Sandeen [Wed, 10 Oct 2012 03:40:11 +0000 (03:40 +0000)]
xfs_io: include headers for preadv/pwritev
We need to include uio.h to avoid:
[CC] pread.o
pread.c: In function `do_pread':
pread.c:198: warning: implicit declaration of function `preadv'
[CC] pwrite.o
pwrite.c: In function `do_pwrite':
pwrite.c:85: warning: implicit declaration of function `pwritev'
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Carlos Maiolino [Mon, 16 Apr 2012 20:56:56 +0000 (20:56 +0000)]
mkfs: Set a clean output in case of invalid inode size
Remove an unnecessary usage() call after a mkfs failure due an invalid inode
size.
A call to usage() at this point confuses the output message which may cause the
user to think it used wrong arguments to mkfs, instead of an invalid inode size.
Signed-off-by: Carlos Maiolino <cmaiolino@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Mike Frysinger [Sat, 25 Aug 2012 23:07:30 +0000 (23:07 +0000)]
libxcmd: link against readline
This library uses readline funcs (the input.c file), so we need to link
this shared library against it.
URL: https://bugs.gentoo.org/432644 Reported-by: David Badia <dbadia@gmail.com> Signed-off-by: Mike Frysinger <vapier@gentoo.org> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Dave Chinner [Wed, 25 Jul 2012 22:30:50 +0000 (22:30 +0000)]
xfs_io: implement pwritev for vectored writes
When looking at KVM based direct IO patterns, I noticed that it was
using preadv and pwritev, and I could not use xfs_io to simulate
these IO patterns. Extend the pwrite command to be able to issue
vectored write IO to enable use to simulate KVM style direct IO.
Also document the new parameters as well as all the missing pwrite
command parameters in the xfs_io(8) man page.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Dave Chinner [Wed, 25 Jul 2012 22:30:49 +0000 (22:30 +0000)]
xfs_io: implement preadv for vectored reads
When looking at KVM based direct IO patterns, I noticed that it was
using preadv and pwritev, and I could not use xfs_io to simulate
these IO patterns. Extend the pread command to be able to issue
vectored read IO to enable use to simulate KVM style direct IO.
Also document the new parameters as well as all the missing pread
command parameters in the xfs_io(8) man page.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Dave Chinner [Wed, 25 Jul 2012 22:30:48 +0000 (22:30 +0000)]
xfs_io: add sync_file_range support
Add sync_file_range support to xfs_io to allow fine grained control
of data writeback and syncing on a given file.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Mark Tinguely <tinguely@sgi.com>
Peter Watkins [Thu, 2 Aug 2012 22:27:16 +0000 (17:27 -0500)]
xfs_db: bmap dump uses wrong btree key/ptr macro
When dumping the bmap with extents in btree form, the traversal
code should use XFS_BMBT_ macros instead of XFS_BMDR_ macros to
access the key and pointer fields below the root node.
Signed-off-by: Peter Watkins <treestem@gmail.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
which does not account for fragmented multiblock dir2.
ndbno was blindly being advanced by m_dirblkfsbs, but then
blkmap_next_off() would return the logical block of the next
mapped extent in blkmap, which may be within the current
(fragmented) dir2 multi-block, not the next multi-block,
because the extent index t hadn't been advanced.
Fix this by calling blkmap_next_off() until ndbno has advanced
into the next multiblock dir2 block, thereby keeping
the extent index t straight while properly advancing
ndbno.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Ben Myers <bpm@sgi.com>
blkmap_next_off() was cryptic (to me), so document what it does.
Also catch cases when the passed in extent index 't' is beyond
the number of extents in the blkmap, so that:
ext = blkmap->exts + *t;
doesn't walk off the end of the array into garbage.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Ben Myers <bpm@sgi.com>
Eric Sandeen [Tue, 5 Jun 2012 18:24:55 +0000 (13:24 -0500)]
xfs_repair: Fix fragmented multiblock dir2 handling in blkmap_getn()
blkmap_getn() contains a loop which populates an array of extents
with mapping information for a dir2 "block," which may consist
of multiple filesystem blocks.
As written, the loop re-allocates the array for each new extent,
leaking the previously allocated memory and worse, losing the
previously filled-in extent information.
Fix this by only allocating the array once, for the maximum
possible number of extents - the number of fs blocks in the dir
block.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Ben Myers <bpm@sgi.com>
Eric Sandeen [Thu, 29 Mar 2012 03:23:11 +0000 (22:23 -0500)]
mkfs.xfs: print std info if agcount makes agsize out of bounds
When specifying a too-small agcount with stripe geometry,
mkfs.xfs can fail with a somewhat unexpected message:
$ mkfs.xfs -f -d file,name=fsfile,size=9764864000b,agcount=31,su=512k,sw=20
Allocation group size (314995613) is not a multiple of the stripe unit (128)
This strikes me as especially odd because normally, mkfs.xfs
tries to fix up the agsize to be a stripe multiple. The only way
we get to the above error message is if ag _size_ is out of bounds;
exiting with an error about alignment rather than about size
seems odd.
Maybe below is too clever, but if by the time we've decided that
agsize is out of bounds after rounding it both up and down,
as necessary, to get to a stripe-width multiple, calling
validate_ag_geometry() will give us the same standard message as
if we had specified no stripe geometry:
$ mkfs/mkfs.xfs -f -d file,name=fsfile,size=9764864000b,agcount=31,su=512k,sw=20
agsize (314995613b) too big, maximum is 268435455 blocks
Usage: mkfs.xfs
...
$ mkfs/mkfs.xfs -f -d file,name=fsfile,size=9764864000b,agcount=31
agsize (314995613b) too big, maximum is 268435455 blocks
Usage: mkfs.xfs
...
Also, tidy up error message to explicitly state "blocks" not "b"
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com>
Eric Sandeen [Thu, 8 Mar 2012 22:45:07 +0000 (16:45 -0600)]
xfs_io: allow -F in open args, remove from help
Now that -F ("foreign") is automagic, we should no longer list
it in the help output for open, but we should still accept
it for compatibility; esp. as it is still in the case statement.
Oops.
Remove the -F option from the manpage open section as well.
Reported-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Mark Tinguely <tinguely@sgi.com>
Link counts are unsigned and need to be printed as such. Also only
print the varning about upgrading the inode version if the inode was
version 1 before.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
It looks like we currently never grow the variable-width nlink array
if only the on-disk nlink size overflows 8 bits. This leads to a major
mess in nlink counting, and eventually an assert in phase7.
Replace the indirect all mess with a union that allows doing proper
array arithmetics while we're at it.
Reviewed-by: Ben Myers <bpm@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
repair: fix incorrect use of thread local data in dir and attr code
The attribute and dirv1 code use pthread thread local data incorrectly in
a few places, which will make them fail in horrible ways when using the
ag_stride options.
Replace the use of thread local data with simple local allocations given
that there is no needed to micro-optimize these allocations as much
as e.g. the extent map. The added benefit is that we have to allocate
less memory, and can free it quickly.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Reported-by: Tom Crane <T.Crane@rhul.ac.uk> Tested-by: Tom Crane <T.Crane@rhul.ac.uk> Signed-off-by: Christoph Hellwig <hch@lst.de>
Make sure we do not reject an XFS root mount just because /dev/root is also
listed in /proc/mounts. The root cause for this was the awkward getmntany
function, which is replaced with a broader reach find_mountpoint function
which replace getmntany and the surrounding code from the main routine in
a structured way. This changes the flow from finding a mounted filesystem
matching the argument and checking that it's XFS to find a mounted XFS
filesystem and thus fixes the bug.
Based on analysis and an earlier patch from
Carlos Maiolino <cmaiolino@redhat.com>.
Reviewed-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Dave Chinner [Fri, 2 Mar 2012 08:34:58 +0000 (08:34 +0000)]
xfs_io: fix fiemap loop continuation
When the fiemap command needs to retrieve more extents from the
kernel via a subsequent IO, it calculates the next logical block to
retrieve in file system block units. the fiemap needs the start
offset in bytes, not filesystem blocks. Hence if the fiemap command
can loop forever retrieving the same blocks if the logical offset
offset of the next block in filesystem block units is smaller than
the number of bytes in a filessytem block. i.e. it will just loop
retreiving the first 32 extents from offset block zero.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Eric Sandeen [Fri, 2 Mar 2012 04:46:35 +0000 (22:46 -0600)]
mkfs.xfs: properly handle physical sector size
This splits the fs_topology structure "sectorsize" into
logical & physical, and gets both via blkid_get_topology().
This primarily allows us to default to using the physical
sectorsize for mkfs's "sector size" value, the fundamental
size of any IOs the filesystem will perform.
We reduce mkfs.xfs's "sector size" to logical if
a block size < physical sector size is specified.
This is suboptimal, but permissable.
For block size < sector size, differentiate the error
message based on whether the sector size was manually
specified, or deduced.
Signed-off-by: Eric Sandeen <sandeen@redhat.com> Reviewed-by: Dave Chinner <dchinner@redhat.com>
Dave Chinner [Thu, 2 Feb 2012 06:21:14 +0000 (17:21 +1100)]
xfs_io: fix fiemap loop continuation
When the fiemap command needs to retrieve more extents from the
kernel via a subsequent IO, it calculates the next logical block to
retrieve in file system block units. the fiemap needs the start
offset in bytes, not filesystem blocks. Hence if the fiemap command
can loop forever retrieving the same blocks if the logical offset
offset of the next block in filesystem block units is smaller than
the number of bytes in a filessytem block. i.e. it will just loop
retreiving the first 32 extents from offset block zero.
Signed-off-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Eric Sandeen [Fri, 27 Jan 2012 19:26:19 +0000 (13:26 -0600)]
xfs_quota: check for size parsing errors
Doing something like
# xfs_quota -x -c 'limit -u bhard=1.2g ...
will cause cvtnum to fail and return a value of -1LL (because it
cannot parse the decimal), but the quota caller doesn't check
for this error value, casts it to U64, shifts right, and we end
up with an answer of 16 petabytes rather than erroring out.
Fix this.
Reviewed-by: Mark Tinguely <tinguely@sgi.com> Reported-by: James Lawrie <james@jdlawrie.co.uk> Signed-off-by: Eric Sandeen <sandeen@redhat.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
repair: update extent count after zapping duplicate blocks
When we find a duplicate extent in an extern format inode we do not zap
the whole inode, but just truncate it to the point where the duplicate
extent was found. But the current code only updates di_nblocks for the
new size, but no di_nextents/di_anextents. In most cases this isn't noticed,
but when moving such an inode to the lost+found directoy the consistency
check in xfs_iformat trips over it. Fix this by updating the on-disk
extent count as part of the inode repair.
Note that we zap btree format inodes with duplicate block completely
at this point, so this fix doesn't apply to them.
Reviewed-by: Mark Tinguely <tinguely@sgi.com> Reported-by: Arkadiusz Mi??kiewicz <arekm@maven.pl> Tested-by: Arkadiusz Mi??kiewicz <arekm@maven.pl> Signed-off-by: Christoph Hellwig <hch@lst.de>
I see "use xfs_repair instead of xfs_check" hint on xfs@irc, mailing
lists and other places but the first source of information (xfs_check
man page) doesn't mention this. Improve that.
Signed-off-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Signed-off-by: Christoph Hellwig <hch@lst.de>
Remove indirections in the inode record bit manipulation macros and flatten
them to a single level of inlines. Also use a common IREC_MASK define
instead of duplicating it for every bitmask.
Reviewed-by: Mark Tinguely <tinguely@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Nathan Scott [Tue, 31 Jan 2012 23:11:16 +0000 (10:11 +1100)]
xfsprogs: extend fiemap configure check
Make the fiemap configure check consistent with the other
libc interface checks - perform a compile and link with a
complete set of symbols, macros and interfaces needed, as
opposed to a build with just the headers.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Nathan Scott <nathans@debian.org> Signed-off-by: Christoph Hellwig <hch@lst.de>
Add a build dependency on linux-libc-dev, to ensure we build
packages with have_fiemap set to true if the headers support
it. Noticed by Dave, some package builds didn't enable this
when they should have.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Nathan Scott <nathans@debian.org> Signed-off-by: Christoph Hellwig <hch@lst.de>
repair: handle filesystems with the log in allocation group 0
Sindre Skogen reported that repair chokes on a very small filesystem created
by mkfs.xfs from xfsprogs 2.9.4. It turned out that for some reason this
filesystem had the log in allocation group 0 and thus repairs validation
of the root inode number was off. Fix this by adding the log blocks if
the log is allocated in allocation group 0.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Reported-by: Sindre Skogen <sindre@workzone.no> Signed-off-by: Christoph Hellwig <hch@lst.de>
The inode prefetching code has a fixed limit of inodes that might are
submitted at a time. Unfortunately the buffers for them get locked
once the prefetching starts. That way the threads processing the inode
might get stuck on buffer locked, but not submitted for reading yet.
Fix this by kicking the queue as soon as we would have to wait on the
ra_count semaphore.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Reported-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Tested-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Signed-off-by: Christoph Hellwig <hch@lst.de>
On a sufficiently corrupt filesystem walking the btree nodes might hit the
same node node again, which currently will deadlock. Use a recursion
counter to avoid the direct deadlock and let them normal loop detection
(two bad nodes and out) do its work. This is how repair behaved before
we added the lock when implementing buffer prefetching.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Reported-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Tested-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Signed-off-by: Christoph Hellwig <hch@lst.de>
repair: allocate and free extent records individually
Instead of allocating inode records in chunks and keeping a freelist of them
which gets released to the system memory allocator in one go use plain malloc
and free for them. The freelist just means adding a global lock instead
of relying on malloc and free which could be implemented lockless. In
addition smart allocators like tcmalloc have far less overhead than our
chunk and linked list.
Reviewed-by: Dave Chinner <dchinner@redhat.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
projects / thirdparty / xfsprogs-dev.git / log
