.\" the Free Software Foundation; either version 2 of the License, or
.\" (at your option) any later version.
.\" See file COPYING in distribution for details.
-.TH MDADM 8 "" v4.0
+.TH MDADM 8 "" v4.1-rc2
.SH NAME
mdadm \- manage MD devices
.I aka
Linear and RAID levels 0/1/4/5/6,
changing the RAID level between 0, 1, 5, and 6, and between 0 and 10,
changing the chunk size and layout for RAID 0,4,5,6,10 as well as adding or
-removing a write-intent bitmap.
+removing a write-intent bitmap and changing the array's consistency policy.
.TP
.B "Incremental Assembly"
which is managed in a similar manner to DDF, and is supported by an
option-rom on some platforms:
.IP
-.B http://www.intel.com/design/chipsets/matrixstorage_sb.htm
+.B https://www.intel.com/content/www/us/en/support/products/122484/memory-and-storage/ssd-software/intel-virtual-raid-on-cpu-intel-vroc.html
.PP
.RE
size, though if there is a variance among the drives of greater than 1%, a warning is
issued.
-A suffix of 'K', 'M' or 'G' can be given to indicate Kilobytes, Megabytes or
-Gigabytes respectively.
+A suffix of 'K', 'M', 'G' or 'T' can be given to indicate Kilobytes,
+Megabytes, Gigabytes or Terabytes respectively.
Sometimes a replacement drive can be a little smaller than the
original drives though this should be minimised by IDEMA standards.
This value can be set with
.B \-\-grow
for RAID level 1/4/5/6 though
-.B CONTAINER
-based arrays such as those with IMSM metadata may not be able to
-support this.
+DDF arrays may not be able to support this.
If the array was created with a size smaller than the currently
active drives, the extra space can be accessed using
.BR \-\-grow .
.B "\-\-grow \-\-array\-size="
command.
-A suffix of 'K', 'M' or 'G' can be given to indicate Kilobytes, Megabytes or
-Gigabytes respectively.
+A suffix of 'K', 'M', 'G' or 'T' can be given to indicate Kilobytes,
+Megabytes, Gigabytes or Terabytes respectively.
A value of
.B max
restores the apparent size of the array to be whatever the real
amount of available space is.
+Clustered arrays do not support this parameter yet.
+
.TP
.BR \-c ", " \-\-chunk=
Specify chunk size of kilobytes. The default when creating an
RAID4, RAID5, RAID6, and RAID10 require the chunk size to be a power
of 2. In any case it must be a multiple of 4KB.
-A suffix of 'K', 'M' or 'G' can be given to indicate Kilobytes, Megabytes or
-Gigabytes respectively.
+A suffix of 'K', 'M', 'G' or 'T' can be given to indicate Kilobytes,
+Megabytes, Gigabytes or Terabytes respectively.
.TP
.BR \-\-rounding=
This option configures the fine details of data layout for RAID5, RAID6,
and RAID10 arrays, and controls the failure modes for
.IR faulty .
+It can also be used for working around a kernel bug with RAID0, but generally
+doesn't need to be used explicitly.
The layout of the RAID5 parity block can be one of
.BR left\-asymmetric ,
"clear" or "none" will remove any pending or periodic failure modes,
and "flush" will clear any persistent faults.
-Finally, the layout options for RAID10 are one of 'n', 'o' or 'f' followed
+The layout options for RAID10 are one of 'n', 'o' or 'f' followed
by a small number. The default is 'n2'. The supported options are:
.I 'n'
number (e.g. it is perfectly legal to have an 'n2' layout for an array
with an odd number of devices).
+A bug introduced in Linux 3.14 means that RAID0 arrays
+.B "with devices of differing sizes"
+started using a different layout. This could lead to
+data corruption. Since Linux 5.4 (and various stable releases that received
+backports), the kernel will not accept such an array unless
+a layout is explictly set. It can be set to
+.RB ' original '
+or
+.RB ' alternate '.
+When creating a new array,
+.I mdadm
+will select
+.RB ' original '
+by default, so the layout does not normally need to be set.
+An array created for either
+.RB ' original '
+or
+.RB ' alternate '
+will not be recognized by an (unpatched) kernel prior to 5.4. To create
+a RAID0 array with devices of differing sizes that can be used on an
+older kernel, you can set the layout to
+.RB ' dangerous '.
+This will use whichever layout the running kernel supports, so the data
+on the array may become corrupt when changing kernel from pre-3.14 to a
+later kernel.
+
When an array is converted between RAID5 and RAID6 an intermediate
RAID6 layout is used in which the second parity block (Q) is always on
the last device. To convert a RAID5 to RAID6 and leave it in this new
.I mdadm
automatically adds an internal bitmap as it will usually be
beneficial. This can be suppressed with
-.B "\-\-bitmap=none".
+.B "\-\-bitmap=none"
+or by selecting a different consistency policy with
+.BR \-\-consistency\-policy .
.TP
.BR \-\-bitmap\-chunk=
bitmap, the chunksize defaults to 64Meg, or larger if necessary to
fit the bitmap into the available space.
-A suffix of 'K', 'M' or 'G' can be given to indicate Kilobytes, Megabytes or
-Gigabytes respectively.
+A suffix of 'K', 'M', 'G' or 'T' can be given to indicate Kilobytes,
+Megabytes, Gigabytes or Terabytes respectively.
.TP
.BR \-W ", " \-\-write\-mostly
which computed a different offset.
Setting the offset explicitly over-rides the default. The value given
-is in Kilobytes unless a suffix of 'K', 'M' or 'G' is used to explicitly
-indicate Kilobytes, Megabytes or Gigabytes respectively.
+is in Kilobytes unless a suffix of 'K', 'M', 'G' or 'T' is used to explicitly
+indicate Kilobytes, Megabytes, Gigabytes or Terabytes respectively.
Since Linux 3.4,
.B \-\-data\-offset
Start the array
.B read only
rather than read-write as normal. No writes will be allowed to the
-array, and no resync, recovery, or reshape will be started.
+array, and no resync, recovery, or reshape will be started. It works with
+Create, Assemble, Manage and Misc mode.
.TP
.BR \-a ", " "\-\-auto{=yes,md,mdp,part,p}{NN}"
Specify journal device for the RAID-4/5/6 array. The journal device
should be a SSD with reasonable lifetime.
+.TP
+.BR \-\-symlinks
+Auto creation of symlinks in /dev to /dev/md, option --symlinks must
+be 'no' or 'yes' and work with --create and --build.
+
+.TP
+.BR \-k ", " \-\-consistency\-policy=
+Specify how the array maintains consistency in case of unexpected shutdown.
+Only relevant for RAID levels with redundancy.
+Currently supported options are:
+.RS
+
+.TP
+.B resync
+Full resync is performed and all redundancy is regenerated when the array is
+started after unclean shutdown.
+
+.TP
+.B bitmap
+Resync assisted by a write-intent bitmap. Implicitly selected when using
+.BR \-\-bitmap .
+
+.TP
+.B journal
+For RAID levels 4/5/6, journal device is used to log transactions and replay
+after unclean shutdown. Implicitly selected when using
+.BR \-\-write\-journal .
+
+.TP
+.B ppl
+For RAID5 only, Partial Parity Log is used to close the write hole and
+eliminate resync. PPL is stored in the metadata region of RAID member drives,
+no additional journal drive is needed.
+
+.PP
+Can be used with \-\-grow to change the consistency policy of an active array
+in some cases. See CONSISTENCY POLICY CHANGES below.
+.RE
+
.SH For assemble:
.BR no\-bitmap ,
.BR bbl ,
.BR no\-bbl ,
+.BR ppl ,
+.BR no\-ppl ,
+.BR layout\-original ,
+.BR layout\-alternate ,
+.BR layout\-unspecified ,
.BR metadata ,
or
.BR super\-minor .
The
.B byteorder
option allows arrays to be moved between machines with different
-byte-order.
+byte-order, such as from a big-endian machine like a Sparc or some
+MIPS machines, to a little-endian x86_64 machine.
When assembling such an array for the first time after a move, giving
.B "\-\-update=byteorder"
will cause
removed. If the bad block list contains entries, this will fail, as
removing the list could cause data corruption.
+The
+.B ppl
+option will enable PPL for a RAID5 array and reserve space for PPL on each
+device. There must be enough free space between the data and superblock and a
+write-intent bitmap or journal must not be used.
+
+The
+.B no\-ppl
+option will disable PPL in the superblock.
+
+The
+.B layout\-original
+and
+.B layout\-alternate
+options are for RAID0 arrays with non-uniform devices size that were in
+use before Linux 5.4. If the array was being used with Linux 3.13 or
+earlier, then to assemble the array on a new kernel,
+.B \-\-update=layout\-original
+must be given. If the array was created and used with a kernel from Linux 3.14 to
+Linux 5.3, then
+.B \-\-update=layout\-alternate
+must be given. This only needs to be given once. Subsequent assembly of the array
+will happen normally.
+For more information, see
+.IR md (4).
+
+The
+.B layout\-unspecified
+option reverts the effect of
+.B layout\-orignal
+or
+.B layout\-alternate
+and allows the array to be again used on a kernel prior to Linux 5.3.
+This option should be used with great caution.
+
.TP
.BR \-\-freeze\-reshape
Option is intended to be used in start-up scripts during initrd boot phase.
.B \-\-continue
option for the grow command.
+.TP
+.BR \-\-symlinks
+See this option under Create and Build options.
+
.SH For Manage mode:
.TP
.TP
.BR \-\-add-journal
-Recreate journal for RAID-4/5/6 array that lost a journal device. In the
-current implementation, this command cannot add a journal to an array
-that had a failed journal. To avoid interrupting on-going write opertions,
+Add journal to an existing array, or recreate journal for RAID-4/5/6 array
+that lost a journal device. To avoid interrupting on-going write opertions,
.B \-\-add-journal
only works for array in Read-Only state.
.TP
.B \-\-examine\-badblocks
List the bad-blocks recorded for the device, if a bad-blocks list has
-been configured. Currently only
+been configured. Currently only
.B 1.x
-metadata supports bad-blocks lists.
+and
+.B IMSM
+metadata support bad-blocks lists.
.TP
.BI \-\-dump= directory
the block where the superblock would be is overwritten even if it
doesn't appear to be valid.
+.B Note:
+Be careful to call \-\-zero\-superblock with clustered raid, make sure
+array isn't used or assembled in other cluster node before execute it.
+
.TP
.B \-\-kill\-subarray=
If the device is a container and the argument to \-\-kill\-subarray
will automatically be added unless some other option is explicitly
requested with the
.B \-\-bitmap
-option. In any case space for a bitmap will be reserved so that one
-can be added layer with
+option or a different consistency policy is selected with the
+.B \-\-consistency\-policy
+option. In any case space for a bitmap will be reserved so that one
+can be added later with
.BR "\-\-grow \-\-bitmap=internal" .
-If the metadata type supports it (currently only 1.x metadata), space
-will be allocated to store a bad block list. This allows a modest
+If the metadata type supports it (currently only 1.x and IMSM metadata),
+space will be allocated to store a bad block list. This allows a modest
number of bad blocks to be recorded, allowing the drive to remain in
service while only partially functional.
.TP
.B \-\-readonly
-start the array readonly \(em not supported yet.
+start the array in readonly mode.
.SH MANAGE MODE
.HP 12
.B \-U
or
.B \-\-update=
-option. Currently only
-.B name
-is supported.
+option. The supported options are
+.BR name ,
+.B ppl
+and
+.BR no\-ppl .
The
.B name
would change the UUID of an active subarray this operation is blocked,
and the command will end in an error.
+The
+.B ppl
+and
+.B no\-ppl
+options enable and disable PPL in the metadata. Currently supported only for
+IMSM subarrays.
+
.TP
.B \-\-examine
The device should be a component of an md array.
.IP \(bu 4
add a write-intent bitmap to any array which supports these bitmaps, or
remove a write-intent bitmap from such an array.
+.IP \(bu 4
+change the array's consistency policy.
.PP
Using GROW on containers is currently supported only for Intel's IMSM
increased - which affects all arrays in the container - or an array
in a container can be converted between levels where those levels are
supported by the container, and the conversion is on of those listed
-above. Resizing arrays in an IMSM container with
-.B "--grow --size"
-is not yet supported.
-
-Grow functionality (e.g. expand a number of raid devices) for Intel's
-IMSM container format has an experimental status. It is guarded by the
-.B MDADM_EXPERIMENTAL
-environment variable which must be set to '1' for a GROW command to
-succeed.
-This is for the following reasons:
-
-.IP 1.
-Intel's native IMSM check-pointing is not fully tested yet.
-This can causes IMSM incompatibility during the grow process: an array
-which is growing cannot roam between Microsoft Windows(R) and Linux
-systems.
-
-.IP 2.
-Interrupting a grow operation is not recommended, because it
-has not been fully tested for Intel's IMSM container format yet.
+above.
.PP
-Note: Intel's native checkpointing doesn't use
+Notes:
+.IP \(bu 4
+Intel's native checkpointing doesn't use
.B --backup-file
option and it is transparent for assembly feature.
+.IP \(bu 4
+Roaming between Windows(R) and Linux systems for IMSM metadata is not
+supported during grow process.
+.IP \(bu 4
+When growing a raid0 device, the new component disk size (or external
+backup size) should be larger than LCM(old, new) * chunk-size * 2,
+where LCM() is the least common multiple of the old and new count of
+component disks, and "* 2" comes from the fact that mdadm refuses to
+use more than half of a spare device for backup space.
.SS SIZE CHANGES
Normally when an array is built the "size" is taken from the smallest
bitmap. If an array has a bitmap, it must be removed before the size
can be changed. Once the change is complete a new bitmap can be created.
+.PP
+Note:
+.B "--grow --size"
+is not yet supported for external file bitmap.
+
.SS RAID\-DEVICES CHANGES
A RAID1 array can work with any number of devices from 1 upwards
.SS CHUNK-SIZE AND LAYOUT CHANGES
-Changing the chunk-size of layout without also changing the number of
+Changing the chunk-size or layout without also changing the number of
devices as the same time will involve re-writing all blocks in-place.
To ensure against data loss in the case of a crash, a
.B --backup-file
in a filesystem that is on the RAID array being affected, the system
will deadlock. The bitmap must be on a separate filesystem.
+.SS CONSISTENCY POLICY CHANGES
+
+The consistency policy of an active array can be changed by using the
+.B \-\-consistency\-policy
+option in Grow mode. Currently this works only for the
+.B ppl
+and
+.B resync
+policies and allows to enable or disable the RAID5 Partial Parity Log (PPL).
+
.SH INCREMENTAL MODE
.HP 12
usually provided by a
.I udev
rules mentioning
-.BR ${DEVLINKS} .
+.BR $env{DEVLINKS} .
.IP +
Does the device have a valid md superblock? If a specific metadata
For further information on mdadm usage, MD and the various levels of
RAID, see:
.IP
-.B http://raid.wiki.kernel.org/
+.B https://raid.wiki.kernel.org/
.PP
(based upon Jakob \(/Ostergaard's Software\-RAID.HOWTO)
.PP
.I mdadm
should always be available from
.IP
-.B http://www.kernel.org/pub/linux/utils/raid/mdadm/
+.B https://www.kernel.org/pub/linux/utils/raid/mdadm/
.PP
Related man pages:
.PP