]>
Commit | Line | Data |
---|---|---|
519561f7 NB |
1 | .\" Copyright Neil Brown and others. |
2 | .\" This program is free software; you can redistribute it and/or modify | |
3 | .\" it under the terms of the GNU General Public License as published by | |
4 | .\" the Free Software Foundation; either version 2 of the License, or | |
5 | .\" (at your option) any later version. | |
6 | .\" See file COPYING in distribution for details. | |
56eb10c0 NB |
7 | .TH MD 4 |
8 | .SH NAME | |
93e790af | 9 | md \- Multiple Device driver aka Linux Software RAID |
56eb10c0 NB |
10 | .SH SYNOPSIS |
11 | .BI /dev/md n | |
12 | .br | |
13 | .BI /dev/md/ n | |
14 | .SH DESCRIPTION | |
15 | The | |
16 | .B md | |
17 | driver provides virtual devices that are created from one or more | |
e0d19036 | 18 | independent underlying devices. This array of devices often contains |
02b76eea NB |
19 | redundancy and the devices are often disk drives, hence the acronym RAID |
20 | which stands for a Redundant Array of Independent Disks. | |
56eb10c0 NB |
21 | .PP |
22 | .B md | |
599e5a36 NB |
23 | supports RAID levels |
24 | 1 (mirroring), | |
25 | 4 (striped array with parity device), | |
26 | 5 (striped array with distributed parity information), | |
27 | 6 (striped array with distributed dual redundancy information), and | |
28 | 10 (striped and mirrored). | |
29 | If some number of underlying devices fails while using one of these | |
98c6faba NB |
30 | levels, the array will continue to function; this number is one for |
31 | RAID levels 4 and 5, two for RAID level 6, and all but one (N-1) for | |
93e790af | 32 | RAID level 1, and dependent on configuration for level 10. |
56eb10c0 NB |
33 | .PP |
34 | .B md | |
e0d19036 | 35 | also supports a number of pseudo RAID (non-redundant) configurations |
570c0542 NB |
36 | including RAID0 (striped array), LINEAR (catenated array), |
37 | MULTIPATH (a set of different interfaces to the same device), | |
38 | and FAULTY (a layer over a single device into which errors can be injected). | |
56eb10c0 | 39 | |
11a3e71d | 40 | .SS MD SUPER BLOCK |
570c0542 NB |
41 | Each device in an array may have a |
42 | .I superblock | |
43 | which records information about the structure and state of the array. | |
44 | This allows the array to be reliably re-assembled after a shutdown. | |
56eb10c0 | 45 | |
570c0542 NB |
46 | From Linux kernel version 2.6.10, |
47 | .B md | |
48 | provides support for two different formats of this superblock, and | |
49 | other formats can be added. Prior to this release, only one format is | |
50 | supported. | |
51 | ||
b3f1c093 | 52 | The common format \(em known as version 0.90 \(em has |
570c0542 | 53 | a superblock that is 4K long and is written into a 64K aligned block that |
11a3e71d | 54 | starts at least 64K and less than 128K from the end of the device |
56eb10c0 NB |
55 | (i.e. to get the address of the superblock round the size of the |
56 | device down to a multiple of 64K and then subtract 64K). | |
11a3e71d | 57 | The available size of each device is the amount of space before the |
56eb10c0 NB |
58 | super block, so between 64K and 128K is lost when a device in |
59 | incorporated into an MD array. | |
93e790af | 60 | This superblock stores multi-byte fields in a processor-dependent |
570c0542 NB |
61 | manner, so arrays cannot easily be moved between computers with |
62 | different processors. | |
63 | ||
b3f1c093 | 64 | The new format \(em known as version 1 \(em has a superblock that is |
570c0542 NB |
65 | normally 1K long, but can be longer. It is normally stored between 8K |
66 | and 12K from the end of the device, on a 4K boundary, though | |
67 | variations can be stored at the start of the device (version 1.1) or 4K from | |
68 | the start of the device (version 1.2). | |
69 | This superblock format stores multibyte data in a | |
93e790af | 70 | processor-independent format and supports up to hundreds of |
570c0542 | 71 | component devices (version 0.90 only supports 28). |
56eb10c0 NB |
72 | |
73 | The superblock contains, among other things: | |
74 | .TP | |
75 | LEVEL | |
11a3e71d | 76 | The manner in which the devices are arranged into the array |
599e5a36 | 77 | (linear, raid0, raid1, raid4, raid5, raid10, multipath). |
56eb10c0 NB |
78 | .TP |
79 | UUID | |
80 | a 128 bit Universally Unique Identifier that identifies the array that | |
93e790af | 81 | contains this device. |
56eb10c0 | 82 | |
2a940e36 NB |
83 | When a version 0.90 array is being reshaped (e.g. adding extra devices |
84 | to a RAID5), the version number is temporarily set to 0.91. This | |
85 | ensures that if the reshape process is stopped in the middle (e.g. by | |
86 | a system crash) and the machine boots into an older kernel that does | |
87 | not support reshaping, then the array will not be assembled (which | |
88 | would cause data corruption) but will be left untouched until a kernel | |
89 | that can complete the reshape processes is used. | |
90 | ||
570c0542 NB |
91 | .SS ARRAYS WITHOUT SUPERBLOCKS |
92 | While it is usually best to create arrays with superblocks so that | |
93e790af SW |
93 | they can be assembled reliably, there are some circumstances when an |
94 | array without superblocks is preferred. These include: | |
570c0542 NB |
95 | .TP |
96 | LEGACY ARRAYS | |
11a3e71d NB |
97 | Early versions of the |
98 | .B md | |
570c0542 NB |
99 | driver only supported Linear and Raid0 configurations and did not use |
100 | a superblock (which is less critical with these configurations). | |
101 | While such arrays should be rebuilt with superblocks if possible, | |
11a3e71d | 102 | .B md |
570c0542 NB |
103 | continues to support them. |
104 | .TP | |
105 | FAULTY | |
106 | Being a largely transparent layer over a different device, the FAULTY | |
107 | personality doesn't gain anything from having a superblock. | |
108 | .TP | |
109 | MULTIPATH | |
110 | It is often possible to detect devices which are different paths to | |
111 | the same storage directly rather than having a distinctive superblock | |
112 | written to the device and searched for on all paths. In this case, | |
113 | a MULTIPATH array with no superblock makes sense. | |
114 | .TP | |
115 | RAID1 | |
116 | In some configurations it might be desired to create a raid1 | |
93e790af | 117 | configuration that does not use a superblock, and to maintain the state of |
addc80c4 NB |
118 | the array elsewhere. While not encouraged for general us, it does |
119 | have special-purpose uses and is supported. | |
11a3e71d | 120 | |
56eb10c0 | 121 | .SS LINEAR |
11a3e71d NB |
122 | |
123 | A linear array simply catenates the available space on each | |
93e790af | 124 | drive to form one large virtual drive. |
11a3e71d NB |
125 | |
126 | One advantage of this arrangement over the more common RAID0 | |
127 | arrangement is that the array may be reconfigured at a later time with | |
93e790af SW |
128 | an extra drive, so the array is made bigger without disturbing the |
129 | data that is on the array. This can even be done on a live | |
11a3e71d NB |
130 | array. |
131 | ||
599e5a36 NB |
132 | If a chunksize is given with a LINEAR array, the usable space on each |
133 | device is rounded down to a multiple of this chunksize. | |
11a3e71d | 134 | |
56eb10c0 | 135 | .SS RAID0 |
11a3e71d NB |
136 | |
137 | A RAID0 array (which has zero redundancy) is also known as a | |
138 | striped array. | |
e0d19036 NB |
139 | A RAID0 array is configured at creation with a |
140 | .B "Chunk Size" | |
c913b90e | 141 | which must be a power of two, and at least 4 kibibytes. |
e0d19036 | 142 | |
2d465520 | 143 | The RAID0 driver assigns the first chunk of the array to the first |
e0d19036 | 144 | device, the second chunk to the second device, and so on until all |
2d465520 | 145 | drives have been assigned one chunk. This collection of chunks forms |
e0d19036 NB |
146 | a |
147 | .BR stripe . | |
93e790af | 148 | Further chunks are gathered into stripes in the same way, and are |
e0d19036 NB |
149 | assigned to the remaining space in the drives. |
150 | ||
2d465520 NB |
151 | If devices in the array are not all the same size, then once the |
152 | smallest device has been exhausted, the RAID0 driver starts | |
e0d19036 NB |
153 | collecting chunks into smaller stripes that only span the drives which |
154 | still have remaining space. | |
155 | ||
156 | ||
56eb10c0 | 157 | .SS RAID1 |
e0d19036 NB |
158 | |
159 | A RAID1 array is also known as a mirrored set (though mirrors tend to | |
5787fa49 | 160 | provide reflected images, which RAID1 does not) or a plex. |
e0d19036 NB |
161 | |
162 | Once initialised, each device in a RAID1 array contains exactly the | |
163 | same data. Changes are written to all devices in parallel. Data is | |
164 | read from any one device. The driver attempts to distribute read | |
165 | requests across all devices to maximise performance. | |
166 | ||
167 | All devices in a RAID1 array should be the same size. If they are | |
168 | not, then only the amount of space available on the smallest device is | |
93e790af | 169 | used (any extra space on other devices is wasted). |
e0d19036 | 170 | |
3dacb890 IP |
171 | Note that the read balancing done by the driver does not make the RAID1 |
172 | performance profile be the same as for RAID0; a single stream of | |
173 | sequential input will not be accelerated (e.g. a single dd), but | |
174 | multiple sequential streams or a random workload will use more than one | |
175 | spindle. In theory, having an N-disk RAID1 will allow N sequential | |
176 | threads to read from all disks. | |
177 | ||
56eb10c0 | 178 | .SS RAID4 |
e0d19036 NB |
179 | |
180 | A RAID4 array is like a RAID0 array with an extra device for storing | |
aa88f531 NB |
181 | parity. This device is the last of the active devices in the |
182 | array. Unlike RAID0, RAID4 also requires that all stripes span all | |
e0d19036 NB |
183 | drives, so extra space on devices that are larger than the smallest is |
184 | wasted. | |
185 | ||
93e790af | 186 | When any block in a RAID4 array is modified, the parity block for that |
e0d19036 NB |
187 | stripe (i.e. the block in the parity device at the same device offset |
188 | as the stripe) is also modified so that the parity block always | |
93e790af | 189 | contains the "parity" for the whole stripe. I.e. its content is |
e0d19036 NB |
190 | equivalent to the result of performing an exclusive-or operation |
191 | between all the data blocks in the stripe. | |
192 | ||
193 | This allows the array to continue to function if one device fails. | |
194 | The data that was on that device can be calculated as needed from the | |
195 | parity block and the other data blocks. | |
196 | ||
56eb10c0 | 197 | .SS RAID5 |
e0d19036 NB |
198 | |
199 | RAID5 is very similar to RAID4. The difference is that the parity | |
200 | blocks for each stripe, instead of being on a single device, are | |
201 | distributed across all devices. This allows more parallelism when | |
93e790af | 202 | writing, as two different block updates will quite possibly affect |
e0d19036 NB |
203 | parity blocks on different devices so there is less contention. |
204 | ||
93e790af | 205 | This also allows more parallelism when reading, as read requests are |
e0d19036 NB |
206 | distributed over all the devices in the array instead of all but one. |
207 | ||
98c6faba NB |
208 | .SS RAID6 |
209 | ||
210 | RAID6 is similar to RAID5, but can handle the loss of any \fItwo\fP | |
211 | devices without data loss. Accordingly, it requires N+2 drives to | |
212 | store N drives worth of data. | |
213 | ||
214 | The performance for RAID6 is slightly lower but comparable to RAID5 in | |
215 | normal mode and single disk failure mode. It is very slow in dual | |
216 | disk failure mode, however. | |
217 | ||
599e5a36 NB |
218 | .SS RAID10 |
219 | ||
93e790af | 220 | RAID10 provides a combination of RAID1 and RAID0, and is sometimes known |
599e5a36 NB |
221 | as RAID1+0. Every datablock is duplicated some number of times, and |
222 | the resulting collection of datablocks are distributed over multiple | |
223 | drives. | |
224 | ||
93e790af | 225 | When configuring a RAID10 array, it is necessary to specify the number |
599e5a36 | 226 | of replicas of each data block that are required (this will normally |
b578481c NB |
227 | be 2) and whether the replicas should be 'near', 'offset' or 'far'. |
228 | (Note that the 'offset' layout is only available from 2.6.18). | |
599e5a36 NB |
229 | |
230 | When 'near' replicas are chosen, the multiple copies of a given chunk | |
231 | are laid out consecutively across the stripes of the array, so the two | |
232 | copies of a datablock will likely be at the same offset on two | |
233 | adjacent devices. | |
234 | ||
235 | When 'far' replicas are chosen, the multiple copies of a given chunk | |
236 | are laid out quite distant from each other. The first copy of all | |
237 | data blocks will be striped across the early part of all drives in | |
238 | RAID0 fashion, and then the next copy of all blocks will be striped | |
239 | across a later section of all drives, always ensuring that all copies | |
240 | of any given block are on different drives. | |
241 | ||
242 | The 'far' arrangement can give sequential read performance equal to | |
91c00388 | 243 | that of a RAID0 array, but at the cost of reduced write performance. |
599e5a36 | 244 | |
b578481c NB |
245 | When 'offset' replicas are chosen, the multiple copies of a given |
246 | chunk are laid out on consecutive drives and at consecutive offsets. | |
247 | Effectively each stripe is duplicated and the copies are offset by one | |
248 | device. This should give similar read characteristics to 'far' if a | |
249 | suitably large chunk size is used, but without as much seeking for | |
250 | writes. | |
251 | ||
599e5a36 | 252 | It should be noted that the number of devices in a RAID10 array need |
93e790af | 253 | not be a multiple of the number of replica of each data block; however, |
599e5a36 NB |
254 | there must be at least as many devices as replicas. |
255 | ||
256 | If, for example, an array is created with 5 devices and 2 replicas, | |
257 | then space equivalent to 2.5 of the devices will be available, and | |
258 | every block will be stored on two different devices. | |
259 | ||
260 | Finally, it is possible to have an array with both 'near' and 'far' | |
93e790af | 261 | copies. If an array is configured with 2 near copies and 2 far |
599e5a36 NB |
262 | copies, then there will be a total of 4 copies of each block, each on |
263 | a different drive. This is an artifact of the implementation and is | |
264 | unlikely to be of real value. | |
265 | ||
bf40ab85 | 266 | .SS MULTIPATH |
e0d19036 NB |
267 | |
268 | MULTIPATH is not really a RAID at all as there is only one real device | |
269 | in a MULTIPATH md array. However there are multiple access points | |
270 | (paths) to this device, and one of these paths might fail, so there | |
271 | are some similarities. | |
272 | ||
a9d69660 | 273 | A MULTIPATH array is composed of a number of logically different |
2d465520 NB |
274 | devices, often fibre channel interfaces, that all refer the the same |
275 | real device. If one of these interfaces fails (e.g. due to cable | |
a9d69660 | 276 | problems), the multipath driver will attempt to redirect requests to |
2d465520 | 277 | another interface. |
e0d19036 | 278 | |
b5e64645 NB |
279 | .SS FAULTY |
280 | The FAULTY md module is provided for testing purposes. A faulty array | |
281 | has exactly one component device and is normally assembled without a | |
282 | superblock, so the md array created provides direct access to all of | |
283 | the data in the component device. | |
284 | ||
285 | The FAULTY module may be requested to simulate faults to allow testing | |
a9d69660 | 286 | of other md levels or of filesystems. Faults can be chosen to trigger |
b5e64645 | 287 | on read requests or write requests, and can be transient (a subsequent |
addc80c4 | 288 | read/write at the address will probably succeed) or persistent |
b5e64645 NB |
289 | (subsequent read/write of the same address will fail). Further, read |
290 | faults can be "fixable" meaning that they persist until a write | |
291 | request at the same address. | |
292 | ||
93e790af | 293 | Fault types can be requested with a period. In this case, the fault |
a9d69660 NB |
294 | will recur repeatedly after the given number of requests of the |
295 | relevant type. For example if persistent read faults have a period of | |
296 | 100, then every 100th read request would generate a fault, and the | |
b5e64645 NB |
297 | faulty sector would be recorded so that subsequent reads on that |
298 | sector would also fail. | |
299 | ||
300 | There is a limit to the number of faulty sectors that are remembered. | |
301 | Faults generated after this limit is exhausted are treated as | |
302 | transient. | |
303 | ||
a9d69660 | 304 | The list of faulty sectors can be flushed, and the active list of |
b5e64645 | 305 | failure modes can be cleared. |
e0d19036 NB |
306 | |
307 | .SS UNCLEAN SHUTDOWN | |
308 | ||
599e5a36 NB |
309 | When changes are made to a RAID1, RAID4, RAID5, RAID6, or RAID10 array |
310 | there is a possibility of inconsistency for short periods of time as | |
93e790af SW |
311 | each update requires at least two block to be written to different |
312 | devices, and these writes probably won't happen at exactly the same | |
599e5a36 NB |
313 | time. Thus if a system with one of these arrays is shutdown in the |
314 | middle of a write operation (e.g. due to power failure), the array may | |
315 | not be consistent. | |
e0d19036 | 316 | |
2d465520 | 317 | To handle this situation, the md driver marks an array as "dirty" |
e0d19036 | 318 | before writing any data to it, and marks it as "clean" when the array |
98c6faba NB |
319 | is being disabled, e.g. at shutdown. If the md driver finds an array |
320 | to be dirty at startup, it proceeds to correct any possibly | |
321 | inconsistency. For RAID1, this involves copying the contents of the | |
322 | first drive onto all other drives. For RAID4, RAID5 and RAID6 this | |
323 | involves recalculating the parity for each stripe and making sure that | |
599e5a36 NB |
324 | the parity block has the correct data. For RAID10 it involves copying |
325 | one of the replicas of each block onto all the others. This process, | |
326 | known as "resynchronising" or "resync" is performed in the background. | |
327 | The array can still be used, though possibly with reduced performance. | |
98c6faba NB |
328 | |
329 | If a RAID4, RAID5 or RAID6 array is degraded (missing at least one | |
93e790af | 330 | drive, two for RAID6) when it is restarted after an unclean shutdown, it cannot |
98c6faba NB |
331 | recalculate parity, and so it is possible that data might be |
332 | undetectably corrupted. The 2.4 md driver | |
e0d19036 | 333 | .B does not |
addc80c4 NB |
334 | alert the operator to this condition. The 2.6 md driver will fail to |
335 | start an array in this condition without manual intervention, though | |
35cc5be4 | 336 | this behaviour can be overridden by a kernel parameter. |
e0d19036 NB |
337 | |
338 | .SS RECOVERY | |
339 | ||
addc80c4 | 340 | If the md driver detects a write error on a device in a RAID1, RAID4, |
599e5a36 NB |
341 | RAID5, RAID6, or RAID10 array, it immediately disables that device |
342 | (marking it as faulty) and continues operation on the remaining | |
93e790af SW |
343 | devices. If there are spare drives, the driver will start recreating |
344 | on one of the spare drives the data which was on that failed drive, | |
599e5a36 NB |
345 | either by copying a working drive in a RAID1 configuration, or by |
346 | doing calculations with the parity block on RAID4, RAID5 or RAID6, or | |
93e790af | 347 | by finding and copying originals for RAID10. |
e0d19036 | 348 | |
addc80c4 NB |
349 | In kernels prior to about 2.6.15, a read error would cause the same |
350 | effect as a write error. In later kernels, a read-error will instead | |
351 | cause md to attempt a recovery by overwriting the bad block. i.e. it | |
352 | will find the correct data from elsewhere, write it over the block | |
353 | that failed, and then try to read it back again. If either the write | |
354 | or the re-read fail, md will treat the error the same way that a write | |
93e790af | 355 | error is treated, and will fail the whole device. |
addc80c4 | 356 | |
2d465520 | 357 | While this recovery process is happening, the md driver will monitor |
e0d19036 NB |
358 | accesses to the array and will slow down the rate of recovery if other |
359 | activity is happening, so that normal access to the array will not be | |
360 | unduly affected. When no other activity is happening, the recovery | |
361 | process proceeds at full speed. The actual speed targets for the two | |
362 | different situations can be controlled by the | |
363 | .B speed_limit_min | |
364 | and | |
365 | .B speed_limit_max | |
366 | control files mentioned below. | |
367 | ||
599e5a36 NB |
368 | .SS BITMAP WRITE-INTENT LOGGING |
369 | ||
370 | From Linux 2.6.13, | |
371 | .I md | |
372 | supports a bitmap based write-intent log. If configured, the bitmap | |
373 | is used to record which blocks of the array may be out of sync. | |
374 | Before any write request is honoured, md will make sure that the | |
375 | corresponding bit in the log is set. After a period of time with no | |
376 | writes to an area of the array, the corresponding bit will be cleared. | |
377 | ||
378 | This bitmap is used for two optimisations. | |
379 | ||
1afe1167 | 380 | Firstly, after an unclean shutdown, the resync process will consult |
599e5a36 | 381 | the bitmap and only resync those blocks that correspond to bits in the |
1afe1167 | 382 | bitmap that are set. This can dramatically reduce resync time. |
599e5a36 NB |
383 | |
384 | Secondly, when a drive fails and is removed from the array, md stops | |
385 | clearing bits in the intent log. If that same drive is re-added to | |
386 | the array, md will notice and will only recover the sections of the | |
387 | drive that are covered by bits in the intent log that are set. This | |
388 | can allow a device to be temporarily removed and reinserted without | |
389 | causing an enormous recovery cost. | |
390 | ||
391 | The intent log can be stored in a file on a separate device, or it can | |
392 | be stored near the superblocks of an array which has superblocks. | |
393 | ||
93e790af | 394 | It is possible to add an intent log to an active array, or remove an |
addc80c4 | 395 | intent log if one is present. |
599e5a36 NB |
396 | |
397 | In 2.6.13, intent bitmaps are only supported with RAID1. Other levels | |
addc80c4 | 398 | with redundancy are supported from 2.6.15. |
599e5a36 NB |
399 | |
400 | .SS WRITE-BEHIND | |
401 | ||
402 | From Linux 2.6.14, | |
403 | .I md | |
addc80c4 | 404 | supports WRITE-BEHIND on RAID1 arrays. |
599e5a36 NB |
405 | |
406 | This allows certain devices in the array to be flagged as | |
407 | .IR write-mostly . | |
408 | MD will only read from such devices if there is no | |
409 | other option. | |
410 | ||
411 | If a write-intent bitmap is also provided, write requests to | |
412 | write-mostly devices will be treated as write-behind requests and md | |
413 | will not wait for writes to those requests to complete before | |
414 | reporting the write as complete to the filesystem. | |
415 | ||
416 | This allows for a RAID1 with WRITE-BEHIND to be used to mirror data | |
8f21823f | 417 | over a slow link to a remote computer (providing the link isn't too |
599e5a36 NB |
418 | slow). The extra latency of the remote link will not slow down normal |
419 | operations, but the remote system will still have a reasonably | |
420 | up-to-date copy of all data. | |
421 | ||
addc80c4 NB |
422 | .SS RESTRIPING |
423 | ||
424 | .IR Restriping , | |
425 | also known as | |
426 | .IR Reshaping , | |
427 | is the processes of re-arranging the data stored in each stripe into a | |
428 | new layout. This might involve changing the number of devices in the | |
93e790af | 429 | array (so the stripes are wider), changing the chunk size (so stripes |
addc80c4 | 430 | are deeper or shallower), or changing the arrangement of data and |
93e790af | 431 | parity (possibly changing the raid level, e.g. 1 to 5 or 5 to 6). |
addc80c4 NB |
432 | |
433 | As of Linux 2.6.17, md can reshape a raid5 array to have more | |
434 | devices. Other possibilities may follow in future kernels. | |
435 | ||
436 | During any stripe process there is a 'critical section' during which | |
35cc5be4 | 437 | live data is being overwritten on disk. For the operation of |
addc80c4 NB |
438 | increasing the number of drives in a raid5, this critical section |
439 | covers the first few stripes (the number being the product of the old | |
440 | and new number of devices). After this critical section is passed, | |
441 | data is only written to areas of the array which no longer hold live | |
b3f1c093 | 442 | data \(em the live data has already been located away. |
addc80c4 NB |
443 | |
444 | md is not able to ensure data preservation if there is a crash | |
445 | (e.g. power failure) during the critical section. If md is asked to | |
446 | start an array which failed during a critical section of restriping, | |
447 | it will fail to start the array. | |
448 | ||
449 | To deal with this possibility, a user-space program must | |
450 | .IP \(bu 4 | |
451 | Disable writes to that section of the array (using the | |
452 | .B sysfs | |
453 | interface), | |
454 | .IP \(bu 4 | |
93e790af | 455 | take a copy of the data somewhere (i.e. make a backup), |
addc80c4 | 456 | .IP \(bu 4 |
93e790af | 457 | allow the process to continue and invalidate the backup and restore |
addc80c4 NB |
458 | write access once the critical section is passed, and |
459 | .IP \(bu 4 | |
93e790af | 460 | provide for restoring the critical data before restarting the array |
addc80c4 NB |
461 | after a system crash. |
462 | .PP | |
463 | ||
464 | .B mdadm | |
93e790af | 465 | versions from 2.4 do this for growing a RAID5 array. |
addc80c4 NB |
466 | |
467 | For operations that do not change the size of the array, like simply | |
468 | increasing chunk size, or converting RAID5 to RAID6 with one extra | |
93e790af SW |
469 | device, the entire process is the critical section. In this case, the |
470 | restripe will need to progress in stages, as a section is suspended, | |
addc80c4 | 471 | backed up, |
93e790af | 472 | restriped, and released; this is not yet implemented. |
addc80c4 NB |
473 | |
474 | .SS SYSFS INTERFACE | |
93e790af | 475 | Each block device appears as a directory in |
addc80c4 | 476 | .I sysfs |
93e790af | 477 | (which is usually mounted at |
addc80c4 NB |
478 | .BR /sys ). |
479 | For MD devices, this directory will contain a subdirectory called | |
480 | .B md | |
481 | which contains various files for providing access to information about | |
482 | the array. | |
483 | ||
484 | This interface is documented more fully in the file | |
485 | .B Documentation/md.txt | |
486 | which is distributed with the kernel sources. That file should be | |
487 | consulted for full documentation. The following are just a selection | |
488 | of attribute files that are available. | |
489 | ||
490 | .TP | |
491 | .B md/sync_speed_min | |
492 | This value, if set, overrides the system-wide setting in | |
493 | .B /proc/sys/dev/raid/speed_limit_min | |
494 | for this array only. | |
495 | Writing the value | |
93e790af SW |
496 | .B "system" |
497 | to this file will cause the system-wide setting to have effect. | |
addc80c4 NB |
498 | |
499 | .TP | |
500 | .B md/sync_speed_max | |
501 | This is the partner of | |
502 | .B md/sync_speed_min | |
503 | and overrides | |
504 | .B /proc/sys/dev/raid/spool_limit_max | |
505 | described below. | |
506 | ||
507 | .TP | |
508 | .B md/sync_action | |
509 | This can be used to monitor and control the resync/recovery process of | |
510 | MD. | |
511 | In particular, writing "check" here will cause the array to read all | |
512 | data block and check that they are consistent (e.g. parity is correct, | |
513 | or all mirror replicas are the same). Any discrepancies found are | |
514 | .B NOT | |
515 | corrected. | |
516 | ||
517 | A count of problems found will be stored in | |
518 | .BR md/mismatch_count . | |
519 | ||
520 | Alternately, "repair" can be written which will cause the same check | |
521 | to be performed, but any errors will be corrected. | |
522 | ||
523 | Finally, "idle" can be written to stop the check/repair process. | |
524 | ||
525 | .TP | |
526 | .B md/stripe_cache_size | |
527 | This is only available on RAID5 and RAID6. It records the size (in | |
528 | pages per device) of the stripe cache which is used for synchronising | |
800053d6 DW |
529 | all write operations to the array and all read operations if the array |
530 | is degraded. The default is 256. Valid values are 17 to 32768. | |
addc80c4 | 531 | Increasing this number can increase performance in some situations, at |
800053d6 DW |
532 | some cost in system memory. Note, setting this value too high can |
533 | result in an "out of memory" condition for the system. | |
534 | ||
535 | memory_consumed = system_page_size * nr_disks * stripe_cache_size | |
addc80c4 | 536 | |
a5ee6dfb DW |
537 | .TP |
538 | .B md/preread_bypass_threshold | |
539 | This is only available on RAID5 and RAID6. This variable sets the | |
540 | number of times MD will service a full-stripe-write before servicing a | |
541 | stripe that requires some "prereading". For fairness this defaults to | |
800053d6 DW |
542 | 1. Valid values are 0 to stripe_cache_size. Setting this to 0 |
543 | maximizes sequential-write throughput at the cost of fairness to threads | |
544 | doing small or random writes. | |
addc80c4 | 545 | |
5787fa49 NB |
546 | .SS KERNEL PARAMETERS |
547 | ||
addc80c4 | 548 | The md driver recognised several different kernel parameters. |
5787fa49 NB |
549 | .TP |
550 | .B raid=noautodetect | |
551 | This will disable the normal detection of md arrays that happens at | |
552 | boot time. If a drive is partitioned with MS-DOS style partitions, | |
553 | then if any of the 4 main partitions has a partition type of 0xFD, | |
554 | then that partition will normally be inspected to see if it is part of | |
555 | an MD array, and if any full arrays are found, they are started. This | |
addc80c4 | 556 | kernel parameter disables this behaviour. |
5787fa49 | 557 | |
a9d69660 NB |
558 | .TP |
559 | .B raid=partitionable | |
560 | .TP | |
561 | .B raid=part | |
562 | These are available in 2.6 and later kernels only. They indicate that | |
563 | autodetected MD arrays should be created as partitionable arrays, with | |
564 | a different major device number to the original non-partitionable md | |
565 | arrays. The device number is listed as | |
566 | .I mdp | |
567 | in | |
568 | .IR /proc/devices . | |
569 | ||
addc80c4 NB |
570 | .TP |
571 | .B md_mod.start_ro=1 | |
572 | This tells md to start all arrays in read-only mode. This is a soft | |
573 | read-only that will automatically switch to read-write on the first | |
574 | write request. However until that write request, nothing is written | |
575 | to any device by md, and in particular, no resync or recovery | |
576 | operation is started. | |
577 | ||
578 | .TP | |
579 | .B md_mod.start_dirty_degraded=1 | |
580 | As mentioned above, md will not normally start a RAID4, RAID5, or | |
581 | RAID6 that is both dirty and degraded as this situation can imply | |
582 | hidden data loss. This can be awkward if the root filesystem is | |
93e790af | 583 | affected. Using this module parameter allows such arrays to be started |
addc80c4 NB |
584 | at boot time. It should be understood that there is a real (though |
585 | small) risk of data corruption in this situation. | |
a9d69660 | 586 | |
5787fa49 NB |
587 | .TP |
588 | .BI md= n , dev , dev ,... | |
a9d69660 NB |
589 | .TP |
590 | .BI md=d n , dev , dev ,... | |
5787fa49 NB |
591 | This tells the md driver to assemble |
592 | .B /dev/md n | |
593 | from the listed devices. It is only necessary to start the device | |
594 | holding the root filesystem this way. Other arrays are best started | |
595 | once the system is booted. | |
596 | ||
a9d69660 NB |
597 | In 2.6 kernels, the |
598 | .B d | |
599 | immediately after the | |
600 | .B = | |
601 | indicates that a partitionable device (e.g. | |
602 | .BR /dev/md/d0 ) | |
603 | should be created rather than the original non-partitionable device. | |
604 | ||
5787fa49 NB |
605 | .TP |
606 | .BI md= n , l , c , i , dev... | |
607 | This tells the md driver to assemble a legacy RAID0 or LINEAR array | |
608 | without a superblock. | |
609 | .I n | |
610 | gives the md device number, | |
611 | .I l | |
612 | gives the level, 0 for RAID0 or -1 for LINEAR, | |
613 | .I c | |
614 | gives the chunk size as a base-2 logarithm offset by twelve, so 0 | |
615 | means 4K, 1 means 8K. | |
616 | .I i | |
617 | is ignored (legacy support). | |
e0d19036 | 618 | |
56eb10c0 NB |
619 | .SH FILES |
620 | .TP | |
621 | .B /proc/mdstat | |
622 | Contains information about the status of currently running array. | |
623 | .TP | |
624 | .B /proc/sys/dev/raid/speed_limit_min | |
93e790af | 625 | A readable and writable file that reflects the current "goal" rebuild |
56eb10c0 NB |
626 | speed for times when non-rebuild activity is current on an array. |
627 | The speed is in Kibibytes per second, and is a per-device rate, not a | |
93e790af | 628 | per-array rate (which means that an array with more disks will shuffle |
56eb10c0 NB |
629 | more data for a given speed). The default is 100. |
630 | ||
631 | .TP | |
632 | .B /proc/sys/dev/raid/speed_limit_max | |
93e790af | 633 | A readable and writable file that reflects the current "goal" rebuild |
56eb10c0 NB |
634 | speed for times when no non-rebuild activity is current on an array. |
635 | The default is 100,000. | |
636 | ||
637 | .SH SEE ALSO | |
638 | .BR mdadm (8), | |
639 | .BR mkraid (8). |