Barry Naujok [Tue, 8 Sep 2009 14:16:56 +0000 (11:16 -0300)]
repair: optimize duplicate extent tracking
Switch the duplicate extent tracking from an avl tree to our new btree
implementation. Modify search_dup_extent to find overlapping extents
with differening start blocks instead of having the caller walk every
possible start block.
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Barry Naujok [Tue, 8 Sep 2009 14:16:44 +0000 (11:16 -0300)]
repair: switch block usage bitmap to a btree
Using a btree representing the extents is much more space efficient than
using a bitmap tracking every single block. In addition it also allows
for more optimal algorithms checking range overlaps instead of walking
every block in various places.
Also move the RT tracking bitmap into incore.c instead of leaving it
a as macros - this keeps the implementation contained.
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Barry Naujok [Tue, 8 Sep 2009 14:16:29 +0000 (11:16 -0300)]
repair: cleanup alloc/free/reset of the block usage tracking
Currently the code to allocate, free and reset the block usage bitmaps
is a complete mess. This patch reorganizes it into logical helpers.
Details:
- the current incore_init code is called just before phase2 is called,
which then marks the log and the AG headers used.
- we get rid of incore_init init, and replace it with direct calls to the
unchanched incore_ino_init/incore_ext_init functions and our new init_bmaps
which does all the allocations for the block usage tracking, aswell
as a call to reset_bmaps to initialize it to the default values.
- reset_bmaps is also called from early phase4 code to reset all state
instead of opencoding it.
- there is a new free_bmaps helper which we call to free our block usage
bitmaps when we don't need them anymore after phase5. The current
code frees some of it a bit early in phase5, but needs to take of it
in phase6 in case we didn't call phase5 due to nomodify mode, and leaks
it if we don't call phase 6, which might happen in case of a bad inode
allocation btree.
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Barry Naujok [Tue, 8 Sep 2009 14:15:44 +0000 (11:15 -0300)]
repair: cleanup helpers for tracking block usage
Rename get_agbno_state/set_agbno_state to get_bmap/set_bmap because
those names are more self-descriptive. Remove the superblous mount
argument to the as the current filesystem is a global in repair.
Remove the fsbno taking variant as they just complicated the code.
Bring all uses of them into the canonical form.
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Barry Naujok [Tue, 8 Sep 2009 14:15:23 +0000 (11:15 -0300)]
repair: track logical to physical block mapping more effeciently
Currently we track the logical to physical block mapping by a structure which
contains an array of physicial blocks. This is extremly efficient and is
replaced with the normal starblock storage we use in the kernel and on disk
in this patch.
In addition also use thread-local storage for the block map, this is possible
because repair only processes one inode at a given time per thread, and the
block map does not have to outlive the processing of a single inode.
The combination of those factors means we can use pthread thread-local
storage to store the block map, and we can re-use the allocation over
and over again.
This should be ported over to xfs_db eventually, or even better we could try
to share the code.
[hch: added a small fix in blkmap_set_ext to not call memmove unless needed]
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Barry Naujok [Tue, 8 Sep 2009 14:14:47 +0000 (11:14 -0300)]
repair: clean up prefetch tracing
Define a dummy pftrace macro for the non-tracing case to reduce the ifdef hell,
clean up a few trace calls and add proper init/exit handlers for the tracing
setup and teardown.
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Barry Naujok [Tue, 8 Sep 2009 14:12:41 +0000 (11:12 -0300)]
repair: use a btree instead of a radix tree for the prefetch queue
Currently the prefetch queue in xfs_repair uses a radix tree implementation
derived from the Linux kernel one to manage it's prefetch queue.
The radix tree implement is not very memory efficient for sparse indices,
so replace it with a btree implementation that is much more efficient.
This is not that important for the prefetch queue but will be very important
for the next memory optimization patches which need a tree to store things
like the block map which are very sparse, and we do not want to deal with
two tree implementations (or rather three given that we still have avl.c
around)
Signed-off-by: Barry Naujok <bnaujok@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Those two functions are almost identical. The big difference is that we only
move blocks from XR_E_FREE1 to XR_E_FREE state when processing the cnt btree.
Besides that we print bno vs cnt in the messages and obviously validate a
slightly different magic number in the header.
The get/put unaligned handlers we use to access the extent descriptor
are not good enough for architectures like Sparc that do not tolerate
dereferencing unaligned pointers. Replace the implementation with the
one the kernel uses on these architectures. It might be a tad
slower on architectures like x86, but I don't want to have multiple
implementations around to not let the testing matrix explode.
Also remove the unaligned.h header which includes another implementation
for unaligned access we don't actually use anymore.
Note that the little change to xfs_inode.c needs to go into the kernel
aswell, I will send a patch for that shortly.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Eric Sandeen <sandeen@sandeen.net> Reported-by: Gabriel Vlasiu <gabrielvlasiu@gmail.com> Tested-by: Gabriel Vlasiu <gabrielvlasiu@gmail.com>
Eric Sandeen [Wed, 26 Aug 2009 23:38:59 +0000 (18:38 -0500)]
xfs_io: actually issue 0 size writes
While testing some stuff in generic_write_checks() in the
kernel I realized that you can't actually use xfs_io to send
a 0-byte write in. This is actually a condition worth testing:
If count is zero and fd refers to a regular file,
then write() may return a failure status if one of
the errors below is detected. If no errors are
detected, 0 will be returned without causing any
other effect.
So fix that up.
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Felix Blyakher <felixb@sgi.com>
Eric Sandeen [Fri, 31 Jul 2009 20:49:49 +0000 (15:49 -0500)]
xfs_db: do bounds checking in frag's scanfunc_bmap
This is for http://oss.sgi.com/bugzilla/show_bug.cgi?id=842
Bug 842 - xfs_db crashes on 'frag'
The nrecs in scanfunc_bmap was corrupted & out of bounds, causing
the loop in process_bmbt_reclist to walk well past allocated memory
and eventually segfault.
Add checking to this scanfunc_bmap() similar to that in similar
functions for check, metadump, and repair.
I'm not sure if we can/should print out any more information here...
# db/xfs_db -r -c frag xfs.img
invalid numrecs (46311) in bmapbtd block
actual 38085, ideal 37731, fragmentation factor 0.93%
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Christoph Hellwig <hch@lst.de>
Eric Sandeen [Thu, 16 Jul 2009 22:00:29 +0000 (17:00 -0500)]
xfs_repair: clear inodes in incorrect btree format
See also RH bug #510823:
https://bugzilla.redhat.com/show_bug.cgi?id=510823
This check in xfs_iformat_btree() tripped:
/*
* blow out if -- fork has less extents than can fit in
* fork (fork shouldn't be a btree format), root btree
* block has more records than can fit into the fork,
* or the number of extents is greater than the number of
* blocks.
*/
leading to:
Jul 10 23:22:45 hermes kernel: Filesystem "dm-11": corrupt inode 2339503222
(btree). Unmount and run xfs_repair.
Jul 10 23:22:45 hermes kernel: Filesystem "dm-11": XFS internal error
xfs_iformat_btree at line 625 of file fs/xfs/xfs_inode.c.
but repair finds nothing at all. xfs_check, however, does flag the inodes
as problematic:
extent count for ino 2339503222 data fork too low (6) for file format
So I copied the xfs_check test into xfs_repair, and voila, it clears
these inodes.
Ideally repair would move these back into extent format, as long as
they are all valid. But I need to look further at that, and for now
there is no other way out of the situation when the kernel finds
the problematic inode, so I'll commit this as a first step.
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Acked-By: Olaf Weber <olaf@sgi.com> Reviewed-by: Christoph Hellwig <hch@infradead.org>
because sb_agcount & sb_agblocks are 32-bit integers; this
may then miss corrupt agbnos for the last ag, which can in
turn lead to out of bounds memory accesses later, for example
when the block nr is used to offset in set_agbno_state():
addr = ba_bmap[(agno)] + (ag_blockno)/XR_BB_NUM;
Similar problems in mk_incore_fstree
Reported-by: Jesse Stroik <jstroik@ssec.wisc.edu> Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Felix Blyakher <felixb@sgi.com>
Eric Sandeen [Mon, 8 Jun 2009 21:39:32 +0000 (16:39 -0500)]
xfs_repair: catch bad depth in traverse_int_dir2block
A bad on-disk tree depth in traverse_int_dir2block() can
later cause a segfault when it's used as an array index in
this function; if we get something beyond the max depth,
just error out and the dir will get rebuilt.
Reported-by: Richard Kolkovich <richard@intrameta.com> Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Christoph Hellwig <hch@lst.de>
Eric Sandeen [Mon, 18 May 2009 16:06:03 +0000 (11:06 -0500)]
xfsprogs: fix readline/editline for xfs_io and xfs_quota
It looks like libxcmd wasn't ever being built with -DENABLE_READLINE
even when it was asked for by configure --enable-readline=yes
so xfs_io & xfs_quota didn't get the functionality.
This seems to fix it up for me (fixes editline too while we're
at it).
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Christoph Hellwig <hch@lst.de>
David Chinner [Sun, 10 May 2009 19:31:22 +0000 (21:31 +0200)]
mkfs: allow to make larger logs
Increase the maximum log size supported by mkfs.
The log size can be increased easily in mkfs by changing a few
defines and a couple of types to allow the log size to increase to
just under 2GB. Theoretically, the log size can be made much, much
larger than this (up to 2^32 sectors), but going beyond 2^31
*bytes* causes integer overflow issues in the kernel log code.
We use a maximum size of just under 2GB as exactly 2GB will cause
overflow issues on the first transaction reservation.
Maximum log size is now reached at a fs size of ~4TB.
Signed-off-by: Dave Chinner <dgc@sgi.com> Signed-off-by: Christoph Hellwig <hch@lst.de>
Currently two xfs_repair processes will happily operate on the same
filesystem device at the same time. It is also possible to mount a
filesystem that is in the process of being repaired.
This patch modifies xfs_repair to open the filesystem devices with
O_EXCL unless it was invoked in "no modify" or "dangerous" mode.
The net effect is that a 2nd xfs_repair will now safely fail with
"xfs_repair: cannot open /dev/foo: Device or resource busy", and a mount
command will fail with (the slightly cryptic) "mount: /dev/foo already
mounted or /mountpoint busy".
Note that this has no effect if the filesystem is stored in a regular
file instead of on a block device.
(Error messages could probably be improved to be more user-friendly in
this new failure case, and it probably wouldn't hurt to add a BLKROGET
ioctl to check for read-only block devices with read-write permissions,
but this does the job for me.)
Signed-off-by: Nathaniel W. Turner <nate@houseofnate.net> Signed-off-by: Christoph Hellwig <hch@lst.de>
We do want to ship the autoconf-generated files over make distclean and
pick them up in Makepkgs so that a user compiling the program doesn't
require autoconf.
For that split up make distclean from make realclean and exclude the
files we want to keep form the former.
To make this easier to maintain adher to stricter rules of use for
CONFIGURE and LDIRT. CONFIGURE now includes all generated files from
autoconf (or for it like our copied install-sh) and gets only removed
on make realclean, but added to LSRCFILES so that Makepkgs can pick it
up. Everything else than needs to be removed on a realclean gets
assigned to LDIRT.
Also split up the configure rule into one to generate the configure
script and one to run it.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mike Frysinger <vapier@gentoo.org>
Remove aclocal.m4 from the repository and generate it when needed.
Move the AC_PROG_LIBTOOL autoconf macro and use libtoolize according to
the libtool info pages.
Make sure that libtoolize adds the auxiliary files (config.guess and config.sub).
Move install-sh into include/ so that libtoolize does not destroy it.
Split up the ``make clean'' and ``make distclean'' targets: the former
removes all files generated during a build. The latter removes all
files generated by libtoolize, autoconf, and configure as well.
Signed-off-by: Andreas Gruenbacher <agruen@suse.de> Signed-off-by: Christoph Hellwig <hch@lst.de>
Currently Makepkgs generates the source tarball as xfsprogs-version.src.tar.gz,
which is not what we used for recent releases and not what most other open
source packages do. Change it to xfsprogs-version.tar.gz, and rename the
binary tarball to xfsprogs-version.bin.tar.gz
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Tomasz Majkowski [Fri, 20 Mar 2009 16:01:58 +0000 (17:01 +0100)]
xfs_io: fix extent array reallocation
The XFS_IOC_FSGETXATTRA ioctl only returns the number of allocated
extents, so when we reallocate the extent array in the bmap command
we have to account for the worst case where there is a whole between
each two allocated extents. Also add some slack to that case to
allow for a file growing while we are racing with it.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Felix Blyakher <felixb@sgi.com>
Mike Frysinger [Tue, 24 Feb 2009 18:53:34 +0000 (19:53 +0100)]
fix parallel builds
I noticed that xfsprogs-3.0.0 sometimes fails to build in parallel with an
error about fsr not being able to find libhandle. Looking at the top level
Makefile shows the obvious missing depend (fsr: libhandle). Playing around
a bit found a few more not so obvious missing depends on the include subdir
so I fixed that too.
Signed-off-by: Mike Frysinger <vapier@gentoo.org> Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
The source file lists in the Makefiles were missing a couple of files.
Because those missing files include the new headers that are only
installed on make install-qa that resulting tarball would not even be
buildable.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Felix Blyakher <felixb@sgi.com>
Jan Engelhardt [Sun, 15 Feb 2009 18:11:27 +0000 (19:11 +0100)]
specfile: remove Distribution tag
There exist a number of rpm specfile tags that make it possible to
mislead users of the produced RPM package, because its presence
overrides anything that has been specified in ~/.rpmmacros (or the
configuration files of the build system used). Such is especially
annoying when Packager:/Vendor: is put in specfiles, because it is
then almost impossible to get the name/email of the person who
_really_ did the packaging (assuming s/he set it in their
.rpmmacros).
Such similarly applies to the Distribution: tag, which is why I
suggest its removal and let the build systems provide the proper
string instead.
Signed-off-by: Jan Engelhardt <jengelh@medozas.de> Reviewed-by: Christoph Hellwig <hch@lst.de>
Eric Sandeen [Tue, 10 Feb 2009 20:41:51 +0000 (14:41 -0600)]
allow xfs_freeze to freeze "foreign" filesystems
Now that the freeze ioctl has been elevated to the VFS,
let's let xfs_io and xfs_freeze freeze other filesystems
as well - add the CMD_FOREIGN_OK flag to freeze & thaw.
I think the worst that will happen on older kernels is that
the command will fail with ENOSYS or EINVAL....
Signed-off-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Christoph Hellwig <hch@lst.de>
Fail gracefully and with a useful error message when we detect an
invalid superblock magic in xfs_db. Add a new -F flag to continue
when seeing the invalid magic number - that way xfs_check and
xfs_metadump can be used on a filesystem that has a corrupted primary
superblock.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Eric Sandeen <sandeen@sandeen.net> Reviewed-by: Felix Blyakher <felixb@sgi.com>
gettext: make build system look more like automake gettext
configure.in: Find all *.c files using find so no need to update
po/Makefile manually (previous method leads to problems like when
someone adds new .c file and forgets to update Makefile - then
translations are removed after update).
po/Makefile: Use automaticly found .c files. Add update-po target.
Show translation statistics when processing po files. Turn on msgfmt
validation (-c).
How to update translations now?
$ ./configure
$ cd po; make update-po
$ vim your.po
$ make
Signed-off-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Reviewed-by: Christoph Hellwig <hch@lst.de>
xfs_quota: add support for passing project paths on cmdline
/etc/project is currently required for using project quota. This
patch adds ability to specify paths at command line.
fs_table_insert_project_path() introduced in libxcmd. Allows to insert
arbitrary path to fs_* iterating engine. We insert our paths with
-1 as project id (because we know no "original" id for this path
and generally this knowledge is not needed).
New option introduced for paths passing - project -p <path>.
Example:
$ sudo rm -f /etc/projects /etc/projid
$ sudo ./xfs_quota -x -c "project -p /tmp/xx -s 60"
Setting up project 60 (path /tmp/xx)...
Processed 1 (/etc/projects and cmdline) paths for project 60 with
recursion depth infinite (-1).
$ sudo ./xfs_quota -x -c "project -p /tmp/xx -c 60"
Checking project 60 (path /tmp/xx)...
Processed 1 (/etc/projects and cmdline) paths for project 60 with
recursion depth infinite (-1).
$ sudo ./xfs_quota -x -c "project -p /tmp/xx -c 70"
Checking project 70 (path /tmp/xx)...
/tmp/xx - project identifier is not set (inode=60, tree=70)
/tmp/xx/file1 - project identifier is not set (inode=60, tree=70)
/tmp/xx/file2 - project identifier is not set (inode=60, tree=70)
/tmp/xx/file3 - project identifier is not set (inode=60, tree=70)
/tmp/xx/dir1 - project identifier is not set (inode=60, tree=70)
/tmp/xx/dir2 - project identifier is not set (inode=60, tree=70)
Processed 1 (/etc/projects and cmdline) paths for project 70 with
recursion depth infinite (-1).
$ sudo ./xfs_quota -x -c "project -p /tmp/xx -s 70"
Setting up project 70 (path /tmp/xx)...
Processed 1 (/etc/projects and cmdline) paths for project 70 with
recursion depth infinite (-1).
$ sudo ./xfs_quota -x -c "project -p /tmp/xx -c 70"
Checking project 70 (path /tmp/xx)...
Processed 1 (/etc/projects and cmdline) paths for project 70 with
recursion depth infinite (-1).
Signed-off-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Reviewed-by: Donald Douwsma <ddouwsma@bigpond.net.au> Reviewed-by: Christoph Hellwig <hch@lst.de>
xfs_quota project directory depth processing can now be limited
to certain depth. For example: xfs_quota -x -c "project -d 2 -s 10"
limits recursion to level 2.
Signed-off-by: Arkadiusz Miśkiewicz <arekm@maven.pl> Reviewed-by: Donald Douwsma <ddouwsma@bigpond.net.au> Reviewed-by: Christoph Hellwig <hch@lst.de>
Nathan Scott [Wed, 28 Jan 2009 08:47:03 +0000 (09:47 +0100)]
rpm spec file update
Since binaries have moved between xfsprogs and xfsdump packages
in version 3.0.0, we need to fix the dependencies in the rpm
specfiles, else rpm will complain.
While in there, remove xfs-cmds Provides/Obsoletes lines, there
is no way anyone is ever going to be requiring those anymore (no
real-world upgrade path having such old skool SGI-released rpms).
Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Nathan Scott [Wed, 28 Jan 2009 08:47:03 +0000 (09:47 +0100)]
rpm spec file update
Since binaries have moved between xfsprogs and xfsdump packages
in version 3.0.0, we need to fix the dependencies in the rpm
specfiles, else rpm will complain.
While in there, remove xfs-cmds Provides/Obsoletes lines, there
is no way anyone is ever going to be requiring those anymore (no
real-world upgrade path having such old skool SGI-released rpms).
Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
There's really no reason to hide fsr in xfsdump, so move it over
to xfsprogs now that we have a 3.0 release with some major shaking
pending. The only code change is to replace the single attr_setf call
with a fsetxattr to avoid a dependency on libattr.
Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Nathan Scott <nscott@aconex.com>
Michael Sinz [Tue, 30 Dec 2008 17:32:51 +0000 (18:32 +0100)]
xfs_check: fix exit codes
xfs_check returns success when the superblock magic is invalid - this makes it
return the serious error exit code (3) for this case, and the invalid arguments
exit code (1) if the arguments to xfs_check were wrong.
A glance at the code suggests that other xfs_check failures appear to return
the correct error codes.
Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Michael Sinz <michael.sinz@sinz.org>
xfs_quota: warn if specified non-zero quota will be round down to zero
User specified quota limit is internally converted to "basic blocks"
unit (512 bytes in size). Quota value will be silently converted
to zero when user enters any value lower than 512 bytes.
Warn in such case:
: # ./xfs_quota -x -c "limit -u bsoft=431 12000" /home
: xfs_quota: Warning: `431' in quota blocks is 0 (unlimited).
Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Eric Sandeen <sandeen@sandeen.net>
Jakub Bogusz [Tue, 30 Dec 2008 17:22:46 +0000 (18:22 +0100)]
Fix build when gettext is enabled
Build fails when gettext is enabled:
progress.c:16: error: initializer element is not constant
progress.c:16: error: (near initialization for ‘rpt_types[0]’)
progress.c:18: error: initializer element is not constant
progress.c:18: error: (near initialization for ‘rpt_types[1]’)
[...]
Fix that using commonly standard N_() macro.
Improvements by Arkadiusz Miśkiewicz <arekm@maven.pl> Reviewed-by: Christoph Hellwig <hch@lst.de>
projects / thirdparty / xfsprogs-dev.git / log
