NeilBrown [Wed, 9 May 2012 10:49:32 +0000 (20:49 +1000)]
New RESHAPE_NO_BACKUP flag to track when backup action is needed.
Some arrays (raid10) never need a backup file, so during assembly
we can avoid the whole Grow_continue check in the case.
Achieve this using a flag set by the metadata handler.
Also get "mdadm -I" to fail if a backup process would be
needed. It currently does fail as the kernel rejects things,
but it is nicer to have this explicit.
NeilBrown [Wed, 9 May 2012 10:49:32 +0000 (20:49 +1000)]
Grow: add raid10 reshape.
RAID10 reshape requires that data_offset be changed.
So we only allow it if the new_data_offset attribute is available,
and we compute a suitable change in data offset.
NeilBrown [Wed, 9 May 2012 10:49:31 +0000 (20:49 +1000)]
Add --data-offset flag for Create and Grow
This can be used to over-ride the automatic assignment of
data offset.
For --create, it is useful to re-create old arrays where different
defaults applied.
For --grow it may be able to force a reshape in the reverse direction.
Spare superblock doesn't depend on other spares in container.
When loading container metadata thunderdome
may pick a small disk for the champion. This will result in incorrect
interpretation of sizes of other disks in container when joint superblock
is returned. If any disk in container has the 2TB attribute set, the result
must have it set too.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
fbdef49811c9e2b54e2064d9af68cfffa77c6e77 incorrectly tried to fix sign
extension of the bitmap offset. However mdinfo->bitmap_offset is a u32
and needs to be converted to a 32 bit signed integer before the sign
extension.
If system is rebooted during rebuild, md driver changes sync_action
from 'recover' to 'idle' (during stopping all md devices).
If mdmon is still running then, it detects the change of sync_action state,
finishes rebuild and writes metadata to disks. After computer's restart
the RAID volume is in Normal state in OROM and rebuild seems to be finished.
After system's start-up RAID volume is in auto-read-only state
and metadata is in Dirty state. Rebuild seems to be finished but it is not.
Data is inconsistent (out-of-sync).
When mdmon detects the change of sync_action from 'recover' to 'idle',
it has to check if rebuild is really finished. Appropriate test was added.
Now mdmon examines each volume's member if it is being rebuilt.
The restriction that --add was not allowed on a device which
looked like a recent member of an array was overly harsh.
The real requirement was to avoid using --add when the array had
failed, and the device being added might contain necessary
information which can only be incorporated by stopping and
re-assembling with --force.
Both --detail and --monitor can report the names of member
devices on an array, and do so by searching /dev and finding
the shortest name that matches.
If
--prefer=foo
is given, they will instead prefer a name that contain /foo/.
So
mdadm --detail /dev/md0 --prefer=by-path
will list the component devices via their /dev/disk/by-path/xxx
names.
When we can for devices using GET_DISK_INFO we currently
limit to 1024. But some arrays can have more than this.
So raise it to 4096 and make the constant a #define.
Adam Kwolek [Fri, 13 Apr 2012 14:52:08 +0000 (16:52 +0200)]
FIX: Assembled second array is in read only state during reshape
When arrays using external metadata are assembled, and one of array
in container is under reshape, second array will remain in read only
state (not auto read only). It is caused by array fact that array
is frozen and mdmon doesn't has opportunity to switch array in r/w mode.
Freezing not reshaped array just after it is being assembled allows mdmon
to enable it for writing.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:52:06 +0000 (16:52 +0200)]
imsm: FIX: Component size alignment check
Put currently existing code for alignment correction in to function
imsm_component_size_aligment_check() and use it for align component size
to chunk size during volume size expansion operation.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:52:02 +0000 (16:52 +0200)]
FIX: Respect metadata size limitations
When reshape_super() updates metadata with new size, due to some metadata
limitations saved value can be different than requested value by user.
Update size (read it from metadata) for setting it in md.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:52:01 +0000 (16:52 +0200)]
FIX: Extend size of raid0 array
For raid0, takeover operation is required for size change.
Add takeover to degraded raid4 before size change and back to raid0 after.
Array information has to be read again from md after takeover.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:51:59 +0000 (16:51 +0200)]
FIX: Support metadata changes rollback
Function reshape_super() guards metadata changes.
It is used to apply changes rollback in error case also.
As change (apply and rollback) can be not bi-directional reshape_super()
has to know if current action is metadata change that should be guarded
using metadata restrictions, or this is metadata rollback change
executed due to error occurrence.
In second case change has to be unconditional.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:51:58 +0000 (16:51 +0200)]
imsm: Execute size change for external metatdata
For external metatdata ioctl doesn't set new size. Set new size using sysfs.
Put code for size change in to function to re-use the same code as during
On-line Capacity Expansion
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:51:56 +0000 (16:51 +0200)]
imsm: FIX: Add volume size expand support to imsm_analyze_change()
Patch adds ability to function imsm_analyze_change() for:
1. Detect size change request for volume operation.
2. Check and correct size for change.
3. Set new change kind to CH_ARRAY_SIZE
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Fri, 13 Apr 2012 14:51:55 +0000 (16:51 +0200)]
imsm: FIX: Update function imsm_num_data_members() for Raid1/10
Function imsm_num_data_members() returns wrong value for raid 1 and 10.
It returns all data member but it should return number of unique data
members (excluding mirror devices)
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
super1: leave more space in front of data by default.
The kernel is growing the ability to avoid the need for a
backup file during reshape by being able to change the data offset.
For this to be useful we need plenty of free space before the
data so the data offset can be reduced.
So for v1.1 and v1.2 metadata make the default data_offset much
larger. Aim for 128Meg, but keep a power of 2 and don't use more
than 0.1% of each device.
Don't change v1.0 as that is used when the data_offset is required to
be zero.
clear hi bits if not used after loading metadata from disk
Functions retrieving sizes from metadata do not need to check
2TB attribute only when we can guarantee the hi bits are always
clear when the MPB_ATTR_2TB_DISK attribute is not set.
Therefore the following fields are cleared on metadata load
when not in use according to attribute:
struct imsm_disk.total_blocks_hi
struct imsm_map.pba_of_lba0_hi
struct imsm_map.blocks_per_member_hi
struct imsm_map.num_data_stripes_hi
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Calculating array_blocks using info->size causes error on activation of
volume using disks over 1 TB. unsigned long long size parameter
is used instead.
total_blocks, pba_of_lba0, blocks_per_member and num_data_stripes overflow
when using disks over 2TB.
Part of fillers in metadata is used to contain hi bits of the numbers
that are likely to go over 32 bit limit.
Functions are added to get and set such fields as the hi bits are not
adjacent with low bits in the structures.
Acked-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
NeilBrown [Wed, 28 Mar 2012 06:29:37 +0000 (17:29 +1100)]
sysfs: fixed sysfs_freeze_array array to work properly with Manage_subdevs.
If the array is already frozen when Manage_subdevs is called we don't
want it to unfreeze the array.
This is because Grow calls Manage_subdevs to add devices to an array
being reshaped, and the array must stay frozen over this call.
So if sysfs_freeze_array find the array to be frozen it returns '0',
meaning that it didn't and cannot freeze it. Then the caller will not
try to unfreeze, which is good.
NeilBrown [Thu, 22 Mar 2012 05:53:51 +0000 (16:53 +1100)]
Create: round off size for RAID1 arrays.
RAID1 arrays don't have a chunk size, but if you ever convert
one to RAID5 you will need at least a small one >= 4K.
So round of size to a multiple of 64K.
This only affect Create, not "--grow --size=max". The latter
is too hard and with smaller returns.
NeilBrown [Thu, 22 Mar 2012 05:15:03 +0000 (16:15 +1100)]
Manage: freeze recovery while adding multiple devices.
If the kernel supports it, freeze recovery over multiple adds,
so that they can all be added to the array at the same time and
be recovered in parallel.
NeilBrown [Thu, 22 Mar 2012 04:34:17 +0000 (15:34 +1100)]
Remove possible crash during RAID6 -> RAID5 reshape.
If a RAID6 array is in a state which doesn't have a
RAID5 equivalent, the code currently dereferences a NULL.
If it does have an equivalent - use that.
If it doesn't but it already in the RAID5-compatible layout
with the Q block last, handle that case,
else require the new layout to be explicitly requested.
NeilBrown [Thu, 22 Mar 2012 03:52:21 +0000 (14:52 +1100)]
Assemble: improve verbose logging when including old devices.
Reporting:
mdadm: added /dev/loop1 to /dev/md0 as 1
mdadm: added /dev/loop2 to /dev/md0 as 2
mdadm: added /dev/loop0 to /dev/md0 as 0
mdadm: /dev/md0 has been started with 2 drives (out of 3).
is confusing - why only 2? Code now reports:
mdadm: added /dev/loop1 to /dev/md0 as 1
mdadm: added /dev/loop2 to /dev/md0 as 2 (possibly out of date)
mdadm: added /dev/loop0 to /dev/md0 as 0
mdadm: /dev/md0 has been started with 2 drives (out of 3).
Jes Sorensen [Tue, 20 Mar 2012 21:00:26 +0000 (08:00 +1100)]
init_super1() memset full buffer allocated for superblock
Avoid possibly using stale data in bitmap and misc area of superblock.
In addition, remove superfluous memsets already covered by memset of
full superblock.
Map file may miss an entry if bad flag is not cleared on update.
This happens for example when an old entry exists in map that
has no mdstat counterpart and we create a new array with the same devnum.
Newly created array will not appear in map if update doesnt clear bad flag.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
NeilBrown [Wed, 7 Mar 2012 04:25:57 +0000 (15:25 +1100)]
Manage: allow --re-add to failed array.
If both "legs" of a RAID1 (or equivalent in RAID10) fail, then one
of the becomes available again it maybe appropriate to re-add the
failed device(s).
So remove the restriction that an array must has 'enough' devices
before being re-added, and if there is no-where to read a superblock
from for matching, then assume the kernel will do necessary checks.
NeilBrown [Tue, 6 Mar 2012 23:47:34 +0000 (10:47 +1100)]
Assemble: support assembling of a RAID0 being reshaped.
This is a bit of a hack and the code need to be made more
general. But this adds the special case of a RAID0 being
reshaped which looks like a RAID4 but doesn't need as many
devices.
NeilBrown [Tue, 6 Mar 2012 23:41:24 +0000 (10:41 +1100)]
Assemble: don't use O_EXCL until we have checked device content.
If we open with O_EXCL before checking that the device is one that
we really want, then that could cause some other process to think
the device is busy when it isn't really.
This particularly affects running "mdadm -A devname" in parallel for
different arrays. One might be looking at a device that it won't
end up using while another trys and fails to look at a device that
it needs.
So delay the O_EXCL until after all identity checks.
Multiple "mdadm -As" will still have races, but that is fundamentally
racy anyway.
Jes Sorensen [Wed, 22 Feb 2012 21:55:19 +0000 (08:55 +1100)]
Print error message if failing to write super for 1.x metadata
In addition remove attempt to print an error message if
write_init_super() fails, as this is handled in the various
write_init_super() functions. This avoids a segfault on error.
Reported by Jim Meyering in
https://bugzilla.redhat.com/show_bug.cgi?id=795461
Jim Meyering [Tue, 21 Feb 2012 12:02:22 +0000 (13:02 +0100)]
avoid double-free upon "old buggy kernel" sysfs_read failure
* Incremental.c (Incremental): On sysfs_read failure, don't call
sysfs_free(sra) just before "goto out_unlock", since that very
same "sra" is freed the same way by the clean-up code below.
Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Thu, 16 Feb 2012 13:16:04 +0000 (14:16 +0100)]
FIX: Changes in '0' case for reshape position verification
Reading sysfs entry that is '0' long should cause an error.
Reshape position cannot be empty.
Absence of reshape position should be ignored. It is possible
that we are about raid0 reshape continuation and it is before takeover.
This means that according metadata (changed by mdmon) it should be reshaped
but md knows nothing about it at this moment. Reshape continuation
in reshape_array() will change it to raid4 and reshape position appears
in sysfs.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Jes Sorensen [Tue, 14 Feb 2012 10:52:13 +0000 (11:52 +0100)]
Use posix_memalign() for memory used to write bitmaps
This makes super[01].c properly align buffers used for the bitmap
using posix_memalign() to make sure the writes don't fail in case the
bitmap is opened using O_DIRECT.
This is based on https://bugzilla.redhat.com/show_bug.cgi?id=789898
and an initial patch by Alexander Murashkin.
Adam Kwolek [Thu, 9 Feb 2012 01:37:40 +0000 (12:37 +1100)]
FIX: restart reshape when reshape process is stopped just between 2 reshapes
When reshape is restarted from '0', very begin of array
it is possible that for external metadata reshape and array
configuration doesn't happen.
Check if md has the same opinion, and reshape is restarted
from 0. If so, this is regular reshape start after reshape
switch in metadata to next array only.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Thu, 9 Feb 2012 01:37:04 +0000 (12:37 +1100)]
imsm: FIX: Clear migration record when migration switches to next volume.
When OLCE is in progress, checkpoint steps are getting bigger due to added space during process.
When mdadm fails after saving "max" to sync_max, mdmon will monitor process
and switch reshape to next array. At this moment we have got information
inconsistency between metadata and migration record.
To avoid this, clear migration record by mdmon /exception from the rule
that migration record is maintained by mdadm/ when reshape switches
to next array.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Tue, 7 Feb 2012 14:03:51 +0000 (15:03 +0100)]
imsm: FIX: Chunk size migration problem
When chunk size migration occurs (e.g. 128k->4k) first checkpoint cannot
be set in md due to too small step. Correct migration record initialization
to allow whole copy area usage and increase migration checkpoint step.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Tue, 7 Feb 2012 14:03:43 +0000 (15:03 +0100)]
Flush mdmon before next reshape step during container operation
Using takeover operation for grow purposes, mdadm has to be sure
that mdmon processes all updates, and if necessary it will be closed
at takeover to raid0 operation. If mdmon is late, next array in container
is processed and due to race condition mdmon closes itself instead to monitor
next reshape operation.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Tue, 7 Feb 2012 14:03:35 +0000 (15:03 +0100)]
Fix: Sometimes mdmon throws core dump during reshape
Problem was found during reshaping 2 volumes /raid0 and raid5/ in container.
Sometimes mdmon throws core dump due to NULL pointer exception.
Problem occurs in scenario:
- managemon: is about spare activation (degraded raid4 volume == raid0 under takeover)
- managemon: detect level change and signals monitor (manage_member() calls replace_array())
- monitor: detects transition raid4/5->raid0 and sets a->container to NULL
to indicate array deactivation
- managemon : continues his work and tries to activate spare (a->check_degraded is set).
NULL pointer is passed to metadata handler activate_spare()
Core dump is generated.
To resolve this situation managemon (after monitor kick) checks again
a->container pointer to learn if current array is not to be deactivated.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
Adam Kwolek [Tue, 7 Feb 2012 14:03:11 +0000 (15:03 +0100)]
imsm: FIX: No new missing disks are allowed during general migration
When during incremental assembly general migration is in progress,
starting degraded array causes that no more disks (even present)
can be added later as array is already started.
Request all previously present disks during general migration for assembly.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
projects / thirdparty / mdadm.git / log
