md.4

   1 .TH MD 4
   2 .SH NAME
   3 md \- Multiple Device driver aka Linux Software Raid
   4 .SH SYNOPSIS
   5 .BI /dev/md n
   6 .br
   7 .BI /dev/md/ n
   8 .SH DESCRIPTION
   9 The
  10 .B md
  11 driver provides virtual devices that are created from one or more
  12 independent underlying devices.  This array of devices often contains
  13 redundancy, and hence the acronym RAID which stands for a Redundant
  14 Array of Independent Devices.
  15 .PP
  16 .B md
  17 supports RAID levels 1 (mirroring) 4 (striped array with parity
  18 device) and 5 (striped array with distributed parity information).
  19 If a single underlying device fails while using one of these levels,
  20 the array will continue to function.
  21 .PP
  22 .B md
  23 also supports a number of pseudo RAID (non-redundant) configurations
  24 including RAID0 (striped array), LINEAR (catenated array) and
  25 MULTIPATH (a set of different interfaces to the same device).
  26
  27 .SS MD SUPER BLOCK
  28 With the exception of Legacy Arrays described below, each device that
  29 is incorporated into an MD array has a
  30 .I super block
  31 written towards the end of the device.  This superblock records
  32 information about the structure and state of the array so that the
  33 array can be reliably re-assembled after a shutdown.
  34
  35 The superblock is 4K long and is written into a 64K aligned block that
  36 starts at least 64K and less than 128K from the end of the device
  37 (i.e. to get the address of the superblock round the size of the
  38 device down to a multiple of 64K and then subtract 64K).
  39 The available size of each device is the amount of space before the
  40 super block, so between 64K and 128K is lost when a device in
  41 incorporated into an MD array.
  42
  43 The superblock contains, among other things:
  44 .TP
  45 LEVEL
  46 The manner in which the devices are arranged into the array
  47 (linear, raid0, raid1, raid4, raid5, multipath).
  48 .TP
  49 UUID
  50 a 128 bit Universally Unique Identifier that identifies the array that
  51 this device is part of.
  52
  53 .SS LEGACY ARRAYS
  54 Early versions of the
  55 .B md
  56 driver only supported Linear and Raid0 configurations and so
  57 did not use an MD superblock (as there is no state that needs to be
  58 recorded).  While it is strongly recommended that all newly created
  59 arrays utilise a superblock to help ensure that they are assembled
  60 properly, the
  61 .B md
  62 driver still supports legacy linear and raid0 md arrays that
  63 do not have a superblock.
  64
  65 .SS LINEAR
  66
  67 A linear array simply catenates the available space on each
  68 drive together to form one large virtual drive.
  69
  70 One advantage of this arrangement over the more common RAID0
  71 arrangement is that the array may be reconfigured at a later time with
  72 an extra drive and so the array is made bigger without disturbing the
  73 data that is on the array.  However this cannot be done on a live
  74 array.
  75
  76
  77 .SS RAID0
  78
  79 A RAID0 array (which has zero redundancy) is also known as a
  80 striped array.
  81 A RAID0 array is configured at creation with a
  82 .B "Chunk Size"
  83 which must be a power of two, and at least 4 kibibytes.
  84
  85 The RAID0 driver assigns the first chunk of the array to the first
  86 device, the second chunk to the second device, and so on until all
  87 drives have been assigned one chunk.  This collection of chunks forms
  88 a
  89 .BR stripe .
  90 Further chunks are gathered into stripes in the same way which are
  91 assigned to the remaining space in the drives.
  92
  93 If devices in the array are not all the same size, then once the
  94 smallest device has been exhausted, the RAID0 driver starts
  95 collecting chunks into smaller stripes that only span the drives which
  96 still have remaining space.
  97
  98
  99 .SS RAID1
 100
 101 A RAID1 array is also known as a mirrored set (though mirrors tend to
 102 provide reflect images, which RAID1 does not) or a plex.
 103
 104 Once initialised, each device in a RAID1 array contains exactly the
 105 same data.  Changes are written to all devices in parallel.  Data is
 106 read from any one device.  The driver attempts to distribute read
 107 requests across all devices to maximise performance.
 108
 109 All devices in a RAID1 array should be the same size.  If they are
 110 not, then only the amount of space available on the smallest device is
 111 used.  Any extra space on other devices is wasted.
 112
 113 .SS RAID4
 114
 115 A RAID4 array is like a RAID0 array with an extra device for storing
 116 parity.  Unlike RAID0, RAID4 also requires that all stripes span all
 117 drives, so extra space on devices that are larger than the smallest is
 118 wasted.
 119
 120 When any block in a RAID4 array is modified the parity block for that
 121 stripe (i.e. the block in the parity device at the same device offset
 122 as the stripe) is also modified so that the parity block always
 123 contains the "parity" for the whole stripe.  i.e. its contents is
 124 equivalent to the result of performing an exclusive-or operation
 125 between all the data blocks in the stripe.
 126
 127 This allows the array to continue to function if one device fails.
 128 The data that was on that device can be calculated as needed from the
 129 parity block and the other data blocks.
 130
 131 .SS RAID5
 132
 133 RAID5 is very similar to RAID4.  The difference is that the parity
 134 blocks for each stripe, instead of being on a single device, are
 135 distributed across all devices.  This allows more parallelism when
 136 writing as two different block updates will quite possibly affect
 137 parity blocks on different devices so there is less contention.
 138
 139 This also allows more parallelism when reading as read requests are
 140 distributed over all the devices in the array instead of all but one.
 141
 142 .SS MUTIPATH
 143
 144 MULTIPATH is not really a RAID at all as there is only one real device
 145 in a MULTIPATH md array.  However there are multiple access points
 146 (paths) to this device, and one of these paths might fail, so there
 147 are some similarities.
 148
 149 A MULTIPATH array is composed of a number of logical different
 150 devices, often fibre channel interfaces, that all refer the the same
 151 real device. If one of these interfaces fails (e.g. due to cable
 152 problems), the multipath driver to attempt to redirect requests to
 153 another interface.
 154
 155
 156 .SS UNCLEAN SHUTDOWN
 157
 158 When changes are made to a RAID1, RAID4, or RAID5 array there is a
 159 possibility of inconsistency for short periods of time as each update
 160 requires are least two block to be written to different devices, and
 161 these writes probably wont happen at exactly the same time.
 162 Thus if a system with one of these arrays is shutdown in the middle of
 163 a write operation (e.g. due to power failure), the array may not be
 164 consistent.
 165
 166 To handle this situation, the md driver marks an array as "dirty"
 167 before writing any data to it, and marks it as "clean" when the array
 168 is being disabled, e.g. at shutdown.
 169 If the md driver finds an array to be dirty at startup, it proceeds to
 170 correct any possibly inconsistency.  For RAID1, this involves copying
 171 the contents of the first drive onto all other drives.
 172 For RAID4 or RAID5 this involves recalculating the parity for each
 173 stripe and making sure that the parity block has the correct data.
 174
 175 If a RAID4 or RAID5 array is degraded (missing one drive) when it is
 176 restarted after an unclean shutdown, it cannot recalculate parity, and
 177 so it is possible that data might be undetectably corrupted.
 178 The md driver currently
 179 .B does not
 180 alert the operator to this condition.  It should probably fail to
 181 start an array in this condition without manual intervention.
 182
 183 .SS RECOVERY
 184
 185 If the md driver detects any error on a device in a RAID1, RAID4, or
 186 RAID5 array, it immediately disables that device (marking it as faulty)
 187 and continues operation on the remaining devices.  If there is a spare
 188 drive, the driver will start recreating on one of the spare drives the
 189 data what was on that failed drive, either by copying a working drive
 190 in a RAID1 configuration, or by doing calculations with the parity
 191 block on RAID4 and RAID5.
 192
 193 While this recovery process is happening, the md driver will monitor
 194 accesses to the array and will slow down the rate of recovery if other
 195 activity is happening, so that normal access to the array will not be
 196 unduly affected.  When no other activity is happening, the recovery
 197 process proceeds at full speed.  The actual speed targets for the two
 198 different situations can be controlled by the
 199 .B speed_limit_min
 200 and
 201 .B speed_limit_max
 202 control files mentioned below.
 203
 204
 205 .SH FILES
 206 .TP
 207 .B /proc/mdstat
 208 Contains information about the status of currently running array.
 209 .TP
 210 .B /proc/sys/dev/raid/speed_limit_min
 211 A readable and writable file that reflects the current goal rebuild
 212 speed for times when non-rebuild activity is current on an array.
 213 The speed is in Kibibytes per second, and is a per-device rate, not a
 214 per-array rate (which means that an array with more disc will shuffle
 215 more data for a given speed).   The default is 100.
 216
 217 .TP
 218 .B /proc/sys/dev/raid/speed_limit_max
 219 A readable and writable file that reflects the current goal rebuild
 220 speed for times when no non-rebuild activity is current on an array.
 221 The default is 100,000.
 222
 223 .SH SEE ALSO
 224 .BR mdadm (8),
 225 .BR mkraid (8).