md.4

   1 .TH MD 4
   2 .SH NAME
   3 md \- Multiple Device driver aka Linux Software Raid
   4 .SH SYNOPSIS
   5 .BI /dev/md n
   6 .br
   7 .BI /dev/md/ n
   8 .SH DESCRIPTION
   9 The
  10 .B md
  11 driver provides virtual devices that are created from one or more
  12 independent underlying devices.  This array of devices often contains
  13 redundancy, and hence the acronym RAID which stands for a Redundant
  14 Array of Independent Devices.
  15 .PP
  16 .B md
  17 supports RAID levels 1 (mirroring) 4 (striped array with parity
  18 device), 5 (striped array with distributed parity information) and 6
  19 (striped array with distributed dual redundancy information.)  If a
  20 some number of underlying devices fails while using one of these
  21 levels, the array will continue to function; this number is one for
  22 RAID levels 4 and 5, two for RAID level 6, and all but one (N-1) for
  23 RAID level 1.
  24 .PP
  25 .B md
  26 also supports a number of pseudo RAID (non-redundant) configurations
  27 including RAID0 (striped array), LINEAR (catenated array) and
  28 MULTIPATH (a set of different interfaces to the same device).
  29
  30 .SS MD SUPER BLOCK
  31 With the exception of Legacy Arrays described below, each device that
  32 is incorporated into an MD array has a
  33 .I super block
  34 written towards the end of the device.  This superblock records
  35 information about the structure and state of the array so that the
  36 array can be reliably re-assembled after a shutdown.
  37
  38 The superblock is 4K long and is written into a 64K aligned block that
  39 starts at least 64K and less than 128K from the end of the device
  40 (i.e. to get the address of the superblock round the size of the
  41 device down to a multiple of 64K and then subtract 64K).
  42 The available size of each device is the amount of space before the
  43 super block, so between 64K and 128K is lost when a device in
  44 incorporated into an MD array.
  45
  46 The superblock contains, among other things:
  47 .TP
  48 LEVEL
  49 The manner in which the devices are arranged into the array
  50 (linear, raid0, raid1, raid4, raid5, multipath).
  51 .TP
  52 UUID
  53 a 128 bit Universally Unique Identifier that identifies the array that
  54 this device is part of.
  55
  56 .SS LEGACY ARRAYS
  57 Early versions of the
  58 .B md
  59 driver only supported Linear and Raid0 configurations and so
  60 did not use an MD superblock (as there is no state that needs to be
  61 recorded).  While it is strongly recommended that all newly created
  62 arrays utilise a superblock to help ensure that they are assembled
  63 properly, the
  64 .B md
  65 driver still supports legacy linear and raid0 md arrays that
  66 do not have a superblock.
  67
  68 .SS LINEAR
  69
  70 A linear array simply catenates the available space on each
  71 drive together to form one large virtual drive.
  72
  73 One advantage of this arrangement over the more common RAID0
  74 arrangement is that the array may be reconfigured at a later time with
  75 an extra drive and so the array is made bigger without disturbing the
  76 data that is on the array.  However this cannot be done on a live
  77 array.
  78
  79
  80 .SS RAID0
  81
  82 A RAID0 array (which has zero redundancy) is also known as a
  83 striped array.
  84 A RAID0 array is configured at creation with a
  85 .B "Chunk Size"
  86 which must be a power of two, and at least 4 kibibytes.
  87
  88 The RAID0 driver assigns the first chunk of the array to the first
  89 device, the second chunk to the second device, and so on until all
  90 drives have been assigned one chunk.  This collection of chunks forms
  91 a
  92 .BR stripe .
  93 Further chunks are gathered into stripes in the same way which are
  94 assigned to the remaining space in the drives.
  95
  96 If devices in the array are not all the same size, then once the
  97 smallest device has been exhausted, the RAID0 driver starts
  98 collecting chunks into smaller stripes that only span the drives which
  99 still have remaining space.
 100
 101
 102 .SS RAID1
 103
 104 A RAID1 array is also known as a mirrored set (though mirrors tend to
 105 provide reflected images, which RAID1 does not) or a plex.
 106
 107 Once initialised, each device in a RAID1 array contains exactly the
 108 same data.  Changes are written to all devices in parallel.  Data is
 109 read from any one device.  The driver attempts to distribute read
 110 requests across all devices to maximise performance.
 111
 112 All devices in a RAID1 array should be the same size.  If they are
 113 not, then only the amount of space available on the smallest device is
 114 used.  Any extra space on other devices is wasted.
 115
 116 .SS RAID4
 117
 118 A RAID4 array is like a RAID0 array with an extra device for storing
 119 parity. This device is the last of the active devices in the
 120 array. Unlike RAID0, RAID4 also requires that all stripes span all
 121 drives, so extra space on devices that are larger than the smallest is
 122 wasted.
 123
 124 When any block in a RAID4 array is modified the parity block for that
 125 stripe (i.e. the block in the parity device at the same device offset
 126 as the stripe) is also modified so that the parity block always
 127 contains the "parity" for the whole stripe.  i.e. its contents is
 128 equivalent to the result of performing an exclusive-or operation
 129 between all the data blocks in the stripe.
 130
 131 This allows the array to continue to function if one device fails.
 132 The data that was on that device can be calculated as needed from the
 133 parity block and the other data blocks.
 134
 135 .SS RAID5
 136
 137 RAID5 is very similar to RAID4.  The difference is that the parity
 138 blocks for each stripe, instead of being on a single device, are
 139 distributed across all devices.  This allows more parallelism when
 140 writing as two different block updates will quite possibly affect
 141 parity blocks on different devices so there is less contention.
 142
 143 This also allows more parallelism when reading as read requests are
 144 distributed over all the devices in the array instead of all but one.
 145
 146 .SS RAID6
 147
 148 RAID6 is similar to RAID5, but can handle the loss of any \fItwo\fP
 149 devices without data loss.  Accordingly, it requires N+2 drives to
 150 store N drives worth of data.
 151
 152 The performance for RAID6 is slightly lower but comparable to RAID5 in
 153 normal mode and single disk failure mode.  It is very slow in dual
 154 disk failure mode, however.
 155
 156 .SS MUTIPATH
 157
 158 MULTIPATH is not really a RAID at all as there is only one real device
 159 in a MULTIPATH md array.  However there are multiple access points
 160 (paths) to this device, and one of these paths might fail, so there
 161 are some similarities.
 162
 163 A MULTIPATH array is composed of a number of logical different
 164 devices, often fibre channel interfaces, that all refer the the same
 165 real device. If one of these interfaces fails (e.g. due to cable
 166 problems), the multipath driver to attempt to redirect requests to
 167 another interface.
 168
 169 .SS FAULTY
 170 The FAULTY md module is provided for testing purposes.  A faulty array
 171 has exactly one component device and is normally assembled without a
 172 superblock, so the md array created provides direct access to all of
 173 the data in the component device.
 174
 175 The FAULTY module may be requested to simulate faults to allow testing
 176 of other md levels or of filesystem.  Faults can be chosen to trigger
 177 on read requests or write requests, and can be transient (a subsequent
 178 read/write at the address will probably succeed) or persistant
 179 (subsequent read/write of the same address will fail).  Further, read
 180 faults can be "fixable" meaning that they persist until a write
 181 request at the same address.
 182
 183 Fault types can be requested with a period.  In this case the fault
 184 will recur repeatedly after the given number of request of the
 185 relevant time.  For example if persistent read faults have a period of
 186 100, then ever 100th read request would generate a fault, and the
 187 faulty sector would be recorded so that subsequent reads on that
 188 sector would also fail.
 189
 190 There is a limit to the number of faulty sectors that are remembered.
 191 Faults generated after this limit is exhausted are treated as
 192 transient.
 193
 194 It list of faulty sectors can be flushed, and the active list of
 195 failure modes can be cleared.
 196
 197 .SS UNCLEAN SHUTDOWN
 198
 199 When changes are made to a RAID1, RAID4, RAID5 or RAID6 array there is a
 200 possibility of inconsistency for short periods of time as each update
 201 requires are least two block to be written to different devices, and
 202 these writes probably wont happen at exactly the same time.
 203 Thus if a system with one of these arrays is shutdown in the middle of
 204 a write operation (e.g. due to power failure), the array may not be
 205 consistent.
 206
 207 To handle this situation, the md driver marks an array as "dirty"
 208 before writing any data to it, and marks it as "clean" when the array
 209 is being disabled, e.g. at shutdown.  If the md driver finds an array
 210 to be dirty at startup, it proceeds to correct any possibly
 211 inconsistency.  For RAID1, this involves copying the contents of the
 212 first drive onto all other drives.  For RAID4, RAID5 and RAID6 this
 213 involves recalculating the parity for each stripe and making sure that
 214 the parity block has the correct data.  This process, known as
 215 "resynchronising" or "resync" is performed in the background.  The
 216 array can still be used, though possibly with reduced performance.
 217
 218 If a RAID4, RAID5 or RAID6 array is degraded (missing at least one
 219 drive) when it is restarted after an unclean shutdown, it cannot
 220 recalculate parity, and so it is possible that data might be
 221 undetectably corrupted.  The 2.4 md driver
 222 .B does not
 223 alert the operator to this condition.  The 2.5 md driver will fail to
 224 start an array in this condition without manual intervention.
 225
 226 .SS RECOVERY
 227
 228 If the md driver detects any error on a device in a RAID1, RAID4,
 229 RAID5 or RAID6 array, it immediately disables that device (marking it
 230 as faulty) and continues operation on the remaining devices.  If there
 231 is a spare drive, the driver will start recreating on one of the spare
 232 drives the data what was on that failed drive, either by copying a
 233 working drive in a RAID1 configuration, or by doing calculations with
 234 the parity block on RAID4, RAID5 or RAID6.
 235
 236 While this recovery process is happening, the md driver will monitor
 237 accesses to the array and will slow down the rate of recovery if other
 238 activity is happening, so that normal access to the array will not be
 239 unduly affected.  When no other activity is happening, the recovery
 240 process proceeds at full speed.  The actual speed targets for the two
 241 different situations can be controlled by the
 242 .B speed_limit_min
 243 and
 244 .B speed_limit_max
 245 control files mentioned below.
 246
 247 .SS KERNEL PARAMETERS
 248
 249 The md driver recognised three different kernel parameters.
 250 .TP
 251 .B raid=noautodetect
 252 This will disable the normal detection of md arrays that happens at
 253 boot time.  If a drive is partitioned with MS-DOS style partitions,
 254 then if any of the 4 main partitions has a partition type of 0xFD,
 255 then that partition will normally be inspected to see if it is part of
 256 an MD array, and if any full arrays are found, they are started.  This
 257 kernel paramenter disables this behaviour.
 258
 259 .TP
 260 .BI md= n , dev , dev ,...
 261 This tells the md driver to assemble
 262 .B /dev/md n
 263 from the listed devices.  It is only necessary to start the device
 264 holding the root filesystem this way.  Other arrays are best started
 265 once the system is booted.
 266
 267 .TP
 268 .BI md= n , l , c , i , dev...
 269 This tells the md driver to assemble a legacy RAID0 or LINEAR array
 270 without a superblock.
 271 .I n
 272 gives the md device number,
 273 .I l
 274 gives the level, 0 for RAID0 or -1 for LINEAR,
 275 .I c
 276 gives the chunk size as a base-2 logarithm offset by twelve, so 0
 277 means 4K, 1 means 8K.
 278 .I i
 279 is ignored (legacy support).
 280
 281 .SH FILES
 282 .TP
 283 .B /proc/mdstat
 284 Contains information about the status of currently running array.
 285 .TP
 286 .B /proc/sys/dev/raid/speed_limit_min
 287 A readable and writable file that reflects the current goal rebuild
 288 speed for times when non-rebuild activity is current on an array.
 289 The speed is in Kibibytes per second, and is a per-device rate, not a
 290 per-array rate (which means that an array with more disc will shuffle
 291 more data for a given speed).   The default is 100.
 292
 293 .TP
 294 .B /proc/sys/dev/raid/speed_limit_max
 295 A readable and writable file that reflects the current goal rebuild
 296 speed for times when no non-rebuild activity is current on an array.
 297 The default is 100,000.
 298
 299 .SH SEE ALSO
 300 .BR mdadm (8),
 301 .BR mkraid (8).