NeilBrown [Thu, 15 May 2014 04:23:16 +0000 (14:23 +1000)]
Grow: store a link to current backup file in /run/mdadm or similar.
Subsequent patch will allow the background part of "mdadm --grow" to
be run from systemd. This can require the passing of a backup file
name.
To do this, store that name as a symlink in /run/mdadm (or MAP_DIR)
and look for it when appropriate.
It might be useful to also store the name across reboot, but that
would be a different patch. We would need to use the uuid to identify
it, and store it in stable storage.
Create: don't default to bitmap=internal when it is not supported
For large arrays (component size > 100GB) if write-intent bitmap is not
enabled, then it is set by default to "internal", even if the metadata
format does support internal bitmaps, which causes Create to fail.
This patch adds checking if add_internal_bitmap is set in the
superswitch before setting bitmap_file to "internal".
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
This modifies locking in Create to eliminate a situation where
--incremental can assemble a device between write_init_super() and
add_disk(), which causes Create to fail.
It sporadically occurs e.g. when metadata is written on a device,
causing an udev change event which triggers mdadm --incremental.
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com> Signed-off-by: NeilBrown <neilb@suse.de>
systemd: various fixes for boot with container-arrays.
1/ Add systemd shutdown script to ensure DDF and IMSM are
clean before we actually shutdown
2/ Get udev to tell systemd to run the mdmon@mdXXX.service
units when a member array appears.
If we boot off a member array (with dracut at least),
the mdmon started in the initramfs will lose track of
/sys etc, so we need to restart it.
systemd will try to forget about it too (but not actually
kill it because we said not to do this).
Having udev tell it to start it will allow a new mdmon to
run which can see /sys, and systemd will know about it.
3/ Always use --offroot and --takeover when starting mdmon with
systemd
--offroot is needed else shutdown will hang.
--takeover is needed incase an mdmon was started earlier
(e.g. in initramfs).
Neither hurt if they aren't actually needed.
DDF: Don't fail compare_super_ddf due to re-configure changes.
It is possible that one device has seem some reconfig but the other
hasn't. In that case they are still the "same" DDF, even though
one might be older. Such age will be detected by 'seq' differences.
If A is new and B is old, then it is import that
mdadm -I B
mdadm -I A
doesn't get confused because A has the same uuid as B, but compare_super fails.
So: if the seq numbers are different, then just accept as two
different superblocks.
If they are the same, then look to copy data from new to old.
DDF: fix possible mdmon crash when updating metadata.
Testing 'c' and then using 'vdc' assumes that the two are in sync,
but sometimes they aren't.
Testing 'vdc' is safer.
This avoids a crash in some cases when failing/removing/added devices
to a DDF.
The stripe locking mechanism must be atomic between
the check and the, potential, autorepair.
For this reason, the autorepair code needs to be just
after the check and both parts (check and autorepair)
must be excuted under stripe lock.
Of course, the manual repair can operate as before.
This patch reorganize the code and provides the single,
atomic, stripe lock.
It should be confirmed that this new locking is not
too demanding.
In case it is, some other solutions will be required
(suggestions wellcome).
Jes Sorensen [Wed, 19 Mar 2014 13:26:02 +0000 (14:26 +0100)]
Work around architectures having statfs.f_type defined as long
Having RAMFS_MAGIC defined as 0x858458f6 causing problems when trying
to compare it directly against statfs.f_type being cast from long to
unsigned long.
This hack is extremly ugly, but it should at least do the right thing
for every situation.
Pawel Baldysiak [Thu, 6 Mar 2014 14:51:44 +0000 (15:51 +0100)]
mdmon@.service: Change type of process start-up to 'forking'.
Mdadm does not wait enough time when mdmon is started by systemd.
It causes various problems with behaviour of a RAID volume with external metadata.
For example: mdmon does not update a value of checkpoint during migration
and second RAID5 volume is read-only after reboot done during
container reshape (both problems occur with IMSM matadata).
If a type of process start-up is changed to 'forking', systemctl will
wait until mdmon (parent) process exits after calling fork.
This way mdmon will always be fully initialized after start_mdmon
and these problems will not occur.
In this case it is recommended to add a path to PIDFile, so that systemd
does not have to guess a PID of the mdmon process.
This patch will remove some legacy code.
It is part of the verbosity "cleanup".
In any case, if information about the P
and Q parity mismatches is required, it
should go inside the code handling page
size blocks, not full stripe size.
NeilBrown [Mon, 20 Jan 2014 22:46:07 +0000 (09:46 +1100)]
systemd/mdmon: set IMSM_NO_PLATFORM=1
As mdmon doesn't inherit environment from mdadm when it is started
by system, it cannot inherit IMSM_NO_PLATFORM.
But if an imsm array as assembled then mdmon really should handle it
whether there is a platform present or not.
So always set this var.
NeilBrown [Mon, 20 Jan 2014 04:31:45 +0000 (15:31 +1100)]
Grow: fix problems with prematurely aborting of reshapes.
1/ when unfreezing, make sure the array is frozen first.
If it isn't we might end up interrupting a reshape.
2/ When the child finishes, don't call abort_reshape() as that
will interrupt the reshape. Just set suspend_* etc
explicitly.
NeilBrown [Mon, 20 Jan 2014 04:27:29 +0000 (15:27 +1100)]
DDF: fix detection of failed devices during assembly.
When we call "getinfo_super", we report the working/failed status
of the particular device, and also (via the 'map') the working/failed
status of every other device that this metadata is aware of.
It is important that the way we calculate "working or failed" is
consistent.
As it is, getinfo_super_ddf() will report a spare as "working", but
every other device will see it as "failed", which leads to failure to
assemble arrays with spares.
For getinfo_super_ddf (i.e. for the container), a device is assumed
"working" unless flagged as DDF_Failed.
For getinfo_super_ddf_bvd (for a member array), a device is assumed
"failed" unless DDF_Online is set, and DDF_Failed is not set.
NeilBrown [Mon, 20 Jan 2014 04:23:31 +0000 (15:23 +1100)]
Assemble: avoid infinite loop when auto-assembling partial container.
When auto-assembling we loop until we get no successes.
If a device is found that look like it is part of an already-existing
container, but we subsequently fail to add that device, then the fact
that the container is running looks like a success. This can result
in infinite looping.
So if a container was already partially assemble, and is still only
partially assembled after we try to add devices, then don't treat that
as success.
Jan Ceuleers [Wed, 11 Dec 2013 07:45:55 +0000 (08:45 +0100)]
Clarify scope of Rebuild events in mdadm manpage
To date, the manpage did not make it clear under which circumstances
Rebuild events are generated, leading to a question on the mailing
list as to whether it is normal for these events to be generated
while checking an array.
So clarify that all operations that act on the entire array are in
scope. The list is given as "e.g.", because it might grow in the
future as other full-array operations are added.
Reported-by: Mark Knecht <markknecht@gmail.com> Signed-off-by: Jan Ceuleers <jan.ceuleers@computer.org> Signed-off-by: NeilBrown <neilb@suse.de>
NeilBrown [Thu, 12 Dec 2013 02:20:32 +0000 (13:20 +1100)]
mdamd-last-resort: add a Conflicts line to stop the timer.
When the md device actually appears we want to stop the timer and not
bother with the mdadm-last-resort@.server. In particular, running
that causes confusing messages and is in general best avoided.
Fortuantely this can simply be achieved with a Conflicts= line
NeilBrown [Wed, 11 Dec 2013 01:29:22 +0000 (12:29 +1100)]
udev rules: try "mdadm -I" on "change" events.
We need to attempt "mdadm -I" on "change" events as well as "add" events,
as the "change" make make a device ready to be part of an array.
This is particularly important for stacked md devices. When the
member devices are "add"ed they don't have any content visible yet.
That doesn't happen until a "change".
NeilBrown [Wed, 11 Dec 2013 01:25:02 +0000 (12:25 +1100)]
udev rules: add some by-pass rules from Fedora
1/ If ANACONDA is running, don't -I assemble any arrays, ANACONDA
needs to be in control
2/ honour "noiswmd" and "nodmraid" kernel command line options.
NeilBrown [Tue, 10 Dec 2013 23:47:54 +0000 (10:47 +1100)]
Add mdmonitor.service systemd unit file.
This systemd unit file runs mdadm in --monitor mode.
It is started by a SYSTEMD_WANTS signal from udev whenever
an md array is started that would benefit from mdadm --monitor.
Commandline arguments can be provided by a script
/usr/lib/systemd/scripts/mdadm_env.sh
which should write an
MDADM_MONITOR_ARGS=....
line to /run/sysconfig/mdadm
A script to extra args from SUSE's /etc/sysconfig/mdadm file
is provided.
If no mdadm_env.sh is provided, then args are "--scan" which
requires "mail" or "program" to be set in /etc/mdadm.conf.
I believe this is suitable for Fedora.
NeilBrown [Tue, 3 Dec 2013 03:01:24 +0000 (14:01 +1100)]
Incremental: improve support for "DEVICE" based restriction in mdadm.conf
--incremental currently fails if the device name passed does not
textually match the names permitted by the DEVICE line in mdadm.conf.
This is problematic when "mdadm -I" is run by udev as the name given
can be a temp name.
This patch makes two improvements:
1/ We generate a list of all existing devices that match the names
in mdadm.conf, and allow rdev based matching
2/ We allows extra aliases to be provided on the command line, and
perform textual matching on those. This is particularly suitable
for udev usages as ${DEVLINKS} can be provided even though the links
make not yet be created.
NeilBrown [Mon, 2 Dec 2013 05:08:04 +0000 (16:08 +1100)]
Systemd integration for starting newly-degraded arrays.
Normally "mdadm -I" will not start an array if it has reason to
expect further devices.
This means that if a device is removed while the host is shut down,
"mdadm -I" will never start the device.
If the array is know to the host, it make sense to start the array
anyway after a reasonable timeout.
This patch adds systemd/udev infrastructure so that 30 seconds after
a known array first becomes able to be assembled as a degraded array,
the array will be assembled even if more devices are still expected.
NeilBrown [Thu, 28 Nov 2013 04:15:30 +0000 (15:15 +1100)]
Incremental: add --export handling.
If --export is given with --incremental, then
MD_DEVNAME
is output which gives the name of the device (in /dev/md) that
is the array (or container) that the device would be added to.
Also
MD_STARTED
is set to one of
no
unsafe
yes
nothing
to indicate if the array was started. IF MD_STARTED=unsafe
then it may be appropriate to run
mdadm -R /dev/md/$MD_DEVNAME
after a timeout to ensure newly degraded array are started.
If
MD_FOREIGN=yes
it might be appropriate to suppress this as the array is
probably not critical.
NeilBrown [Thu, 28 Nov 2013 02:30:23 +0000 (13:30 +1100)]
Incremental: remove test that can never succeed.
Incremental_container never returns 1, so this test is pointless.
It is a holdover from when we called "Incremental()" rather than
"Incremental_container()" at this point.
NeilBrown [Tue, 19 Nov 2013 23:49:14 +0000 (10:49 +1100)]
IMSM metadata really should be ignored when found on partitions.
commit b31df43682216d1c65813eae49ebdd8253db8907
changed load_super_imsm to not insist on finding a partition if
ignore_hw_compat was set.
Unfortunately this is set for '--assemble' so arrays could get
assembled badly.
The comment says this was to allow e.g. --examine of image files.
A better fixes for this is to change test_partitions to not report
a regular file as being a partition.
The errors from the BLKPG ioctl are:
ENOTTY : not a block device.
EINVAL : not a whole device (probably a partition)
ENXIO : partition doesn't exist (so not a partition)
Monitor: write meta data in readonly state, sometimes
This patch reverts 24a216bf:
"Monitor: Don't write metadata in inactive array state".
While it's true that writing meta data is usually not necessary
in readonly state, there is one important exception: if a
disk goes faulty, we want to record that, even if the array is
inactive.
We might as well just revert 24a216bf, because with the recently
submitted patch
"Monitor: don't set arrays dirty after transition to read-only"
those meta data writes that really annoying (for a clean, readonly,
healthy array during startup) are gone anyway.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
tests/10ddf-incremental-wrong-order: new unit test
This is a test simulating two temporary missing disks. These will
have less recent meta data than the other disks in the container.
When the array is reassembled, we expect mdadm to detect that
and react to it by using the meta data of the more recent disks
as reference.
This test FAILS with mdadm 3.3 for DDF.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
Monitor: don't set arrays dirty after transition to read-only
This patch reverts commit 4867e068. Setting arrays dirty after
transition from inactive to anything else causes unnecessary
meta data writes and may wreak trouble unnecessarily when
a disk was missing during assembly but the array was never
written to.
The reason for 4867e068 was a special situation during reshape
from RAID0 to RAID4. I ran all IMSM test cases with it reverted
and found no regressions, so I believe the reshape logic for
IMSM works fine in mdadm 3.3 also without this.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
Jes Sorensen [Thu, 10 Oct 2013 07:42:05 +0000 (09:42 +0200)]
Be consistent in return types from byteswap macros
The bswap_*() macros return int values. Make sure we return the
equivalent types in same byteorder pass-through functions to avoid
problems with the original type leaking through to printf() etc.
Michael Tokarev [Tue, 17 Sep 2013 16:20:40 +0000 (20:20 +0400)]
Remove bashism from Makefile
Makefile uses [ x == y ] construct which does not work
with POSIX shell. Since this is just testing a flag,
replace it with string comparison (=) operator instead.
Signed-off-By: Michael Tokarev <mjt@tls.msk.ru> Signed-off-by: NeilBrown <neilb@suse.de>
commit 3e32ba9d removed support for --offroot, and a9c15847 made
mdmon use @ in argv[0] only when started from initrd.
This breaks mdadm in OpenSUSE 12.3, which starts mdmon from the
root file system and relies on --offroot to work as documented earlier.
Reintroducing --offroot as an undocumented option, as its use is going to
go away soon anyway.
If this can't be applied, it should probably be included as distro-specific
patch if mdadm 3.3 is built for OpenSUSE 12.3. I haven't checked if the
patch is necesary for OpenSUSE Factory, too.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
config: set "auto_seen" after processing the auto line.
Otherwise when we process an empty autoline (to be sure to
capture the MDADM_CONF_AUTO environment variable) we can end up
setting everything to 'yes' which over-rides 'no'.
DDF: handle fake RAIDs with changing subarray UUIDs
Some fake RAID BIOSes (in particular, LSI ones) change the
VD GUID at every boot. These GUIDs are not suitable for
identifying an array. Luckily the header GUID appears to
remain constant.
We construct a pseudo-UUID from the header GUID and those
properties of the subarray that we expect to remain constant.
This is only array name and index; all else might change e.g.
during grow.
Don't do this for all non-MD arrays, only for those known
to use varying volume GUIDs.
This patch obsoletes my previous patch "DDF: new algorithm
for subarray UUID"
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
If a distribution allows the choice between using mdadm and
dmraid for DDF and IMSM to be made by some config file
(/etc/defaults/ /sys/sysconfig/ etc) which is queried by
/etc/init.d scripts, then the fact that mdadm implements this
choce through the config file is not very helpful.
So allow the "AUTO" line to be specified in part using MDADM_CONF_AUTO
in environment.
Config: multiple occurences of lines is not an error.
As we now support config directories it is helpful if
lines are allowed to occur multiple times with one
over-riding the other.
So stop giving warnings when later lines are ignored.
config: read /etc/mdadm.conf.d as well as /etc/mdadm.conf
If a configfile is explicitly given, just that file or directory
is read. Otherwise we now read both a file
/etc/mdadm.conf
and a directory
/etc/mdadm.conf.d
This allows a transition to directory based config, which in turn
allows easy control from scripts.
If config file is a directory, process each file within with a name
ending in ".conf" that doesn't start with ".".
Files are processed in lexical order.
Without this, array may not go clean and mdmon will then
not exit.
A safe_mode of '0' (which is the only one that is handled differently
by this patch) means "never switch to 'active_idle'". We don't want
that when mdmon is stopping.
Assemble: don't ever consider a 'spare' to be the 'most recent'.
If all devices have the same event count and the first one is a spare,
then that spare will be the 'most_recent'.
However then other devices will think the 'most_recent' has failed
(for v0.90 metadata) and will be rejected from the array.
So never consider a 'spare' to be 'most recent'.
Reported-by: Andreas Baer <synthetic.gods@gmail.com> Signed-off-by: NeilBrown <neilb@suse.de>
mwilck@arcor.de [Fri, 16 Aug 2013 18:21:58 +0000 (20:21 +0200)]
DDF: increase default value for safe_mode_delay to 4000ms
That is the same value that IMSM uses. The current default of 200ms
seems to have been copied from the native MD meta data. That value
appears to be much too low for DDF, given that writing the DDF meta
data means that easily several MB worth of data need to be written to
disk.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
mwilck@arcor.de [Fri, 16 Aug 2013 18:21:57 +0000 (20:21 +0200)]
DDF: container_content_ddf: set safe_mode_delay > 0
Set safe_mode_delay to something >0, otherwise all container subarrays
assembled will have safe_mode_delay=0. That will break the assumption that
meta data becomes clean after running mdadm --wait-clean.
Use the same value as in getinfo_super_ddf_bvd. It would be cleaner
to call that directly from container_content_ddf, but I need to check
possible side effects first.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
At this point 'di' and 'rv' both have the same value. gcc doesn't
realise that and a human reader might not either.
'rv' makes more sense too, so use that.
Signed-off-by: Martin Wilck <mwilck@arcor.de> Signed-off-by: NeilBrown <neilb@suse.de>
projects / thirdparty / mdadm.git / log
