mdmon.8

   1 .\" See file COPYING in distribution for details.
   2 .TH MDMON 8 "" v3.2.5
   3 .SH NAME
   4 mdmon \- monitor MD external metadata arrays
   5
   6 .SH SYNOPSIS
   7
   8 .BI mdmon " [--all] [--takeover] CONTAINER"
   9
  10 .SH OVERVIEW
  11 The 2.6.27 kernel brings the ability to support external metadata arrays.
  12 External metadata implies that user space handles all updates to the metadata.
  13 The kernel's responsibility is to notify user space when a "metadata event"
  14 occurs, like disk failures and clean-to-dirty transitions.  The kernel, in
  15 important cases, waits for user space to take action on these notifications.
  16
  17 .SH DESCRIPTION
  18 .SS Metadata updates:
  19 To service metadata update requests a daemon,
  20 .IR mdmon ,
  21 is introduced.
  22 .I Mdmon
  23 is tasked with polling the sysfs namespace looking for changes in
  24 .BR array_state ,
  25 .BR sync_action ,
  26 and per disk
  27 .BR state
  28 attributes.  When a change is detected it calls a per metadata type
  29 handler to make modifications to the metadata.  The following actions
  30 are taken:
  31 .RS
  32 .TP
  33 .B array_state \- inactive
  34 Clear the dirty bit for the volume and let the array be stopped
  35 .TP
  36 .B array_state \- write pending
  37 Set the dirty bit for the array and then set
  38 .B array_state
  39 to
  40 .BR active .
  41 Writes
  42 are blocked until userspace writes
  43 .BR active.
  44 .TP
  45 .B array_state \- active-idle
  46 The safe mode timer has expired so set array state to clean to block writes to the array
  47 .TP
  48 .B array_state \- clean
  49 Clear the dirty bit for the volume
  50 .TP
  51 .B array_state \- read-only
  52 This is the initial state that all arrays start at.
  53 .I mdmon
  54 takes one of the three actions:
  55 .RS
  56 .TP
  57 1/
  58 Transition the array to read-auto keeping the dirty bit clear if the metadata
  59 handler determines that the array does not need resyncing or other modification
  60 .TP
  61 2/
  62 Transition the array to active if the metadata handler determines a resync or
  63 some other manipulation is necessary
  64 .TP
  65 3/
  66 Leave the array read\-only if the volume is marked to not be monitored; for
  67 example, the metadata version has been set to "external:\-dev/md127" instead of
  68 "external:/dev/md127"
  69 .RE
  70 .TP
  71 .B sync_action \- resync\-to\-idle
  72 Notify the metadata handler that a resync may have completed.  If a resync
  73 process is idled before it completes this event allows the metadata handler to
  74 checkpoint resync.
  75 .TP
  76 .B sync_action \- recover\-to\-idle
  77 A spare may have completed rebuilding so tell the metadata handler about the
  78 state of each disk.  This is the metadata handler's opportunity to clear
  79 any "out-of-sync" bits and clear the volume's degraded status.  If a recovery
  80 process is idled before it completes this event allows the metadata handler to
  81 checkpoint recovery.
  82 .TP
  83 .B <disk>/state \- faulty
  84 A disk failure kicks off a series of events.  First, notify the metadata
  85 handler that a disk has failed, and then notify the kernel that it can unblock
  86 writes that were dependent on this disk.  After unblocking the kernel this disk
  87 is set to be removed+ from the member array.  Finally the disk is marked failed
  88 in all other member arrays in the container.
  89 .IP
  90 + Note This behavior differs slightly from native MD arrays where
  91 removal is reserved for a
  92 .B mdadm --remove
  93 event.  In the external metadata case the container holds the final
  94 reference on a block device and a
  95 .B mdadm --remove <container> <victim>
  96 call is still required.
  97 .RE
  98
  99 .SS Containers:
 100 .P
 101 External metadata formats, like DDF, differ from the native MD metadata
 102 formats in that they define a set of disks and a series of sub-arrays
 103 within those disks.  MD metadata in comparison defines a 1:1
 104 relationship between a set of block devices and a raid array.  For
 105 example to create 2 arrays at different raid levels on a single
 106 set of disks, MD metadata requires the disks be partitioned and then
 107 each array can be created with a subset of those partitions.  The
 108 supported external formats perform this disk carving internally.
 109 .P
 110 Container devices simply hold references to all member disks and allow
 111 tools like
 112 .I mdmon
 113 to determine which active arrays belong to which
 114 container.  Some array management commands like disk removal and disk
 115 add are now only valid at the container level.  Attempts to perform
 116 these actions on member arrays are blocked with error messages like:
 117 .IP
 118 "mdadm: Cannot remove disks from a \'member\' array, perform this
 119 operation on the parent container"
 120 .P
 121 Containers are identified in /proc/mdstat with a metadata version string
 122 "external:<metadata name>". Member devices are identified by
 123 "external:/<container device>/<member index>", or "external:-<container
 124 device>/<member index>" if the array is to remain readonly.
 125
 126 .SH OPTIONS
 127 .TP
 128 CONTAINER
 129 The
 130 .B container
 131 device to monitor.  It can be a full path like /dev/md/container, or a
 132 simple md device name like md127.
 133 .TP
 134 .B \-\-takeover
 135 This instructs
 136 .I mdmon
 137 to replace any active
 138 .I mdmon
 139 which is currently monitoring the array.  This is primarily used late
 140 in the boot process to replace any
 141 .I mdmon
 142 which was started from an
 143 .B initramfs
 144 before the root filesystem was mounted.  This avoids holding a
 145 reference on that
 146 .B initramfs
 147 indefinitely and ensures that the
 148 .I pid
 149 and
 150 .I sock
 151 files used to communicate with
 152 .I mdmon
 153 are in a standard place.
 154 .TP
 155 .B \-\-all
 156 This tells mdmon to find any active containers and start monitoring
 157 each of them if appropriate.  This is normally used with
 158 .B \-\-takeover
 159 late in the boot sequence.
 160 A separate
 161 .I mdmon
 162 process is started for each container as the
 163 .B \-\-all
 164 argument is over-written with the name of the container.  To allow for
 165 containers with names longer than 5 characters, this argument can be
 166 arbitrarily extended, e.g. to
 167 .BR \-\-all-active-arrays .
 168 .TP
 169
 170 .PP
 171 Note that
 172 .I mdmon
 173 is automatically started by
 174 .I mdadm
 175 when needed and so does not need to be considered when working with
 176 RAID arrays.  The only times it is run other than by
 177 .I  mdadm
 178 is when the boot scripts need to restart it after mounting the new
 179 root filesystem.
 180
 181 .SH START UP AND SHUTDOWN
 182
 183 As
 184 .I mdmon
 185 needs to be running whenever any filesystem on the monitored device is
 186 mounted there are special considerations when the root filesystem is
 187 mounted from an
 188 .I mdmon
 189 monitored device.
 190 Note that in general
 191 .I mdmon
 192 is needed even if the filesystem is mounted read-only as some
 193 filesystems can still write to the device in those circumstances, for
 194 example to replay a journal after an unclean shutdown.
 195
 196 When the array is assembled by the
 197 .B initramfs
 198 code, mdadm will automatically start
 199 .I mdmon
 200 as required.  This means that
 201 .I mdmon
 202 must be installed on the
 203 .B initramfs
 204 and there must be a writable filesystem (typically tmpfs) in which
 205 .B mdmon
 206 can create a
 207 .B .pid
 208 and
 209 .B .sock
 210 file.  The particular filesystem to use is given to mdmon at compile
 211 time and defaults to
 212 .BR /run/mdadm .
 213
 214 This filesystem must persist through to shutdown time.
 215
 216 After the final root filesystem has be instantiated (usually with
 217 .BR pivot_root )
 218 .I mdmon
 219 should be run with
 220 .I "\-\-all \-\-takeover"
 221 so that the
 222 .I mdmon
 223 running from the
 224 .B initramfs
 225 can be replaced with one running in the main root, and so the
 226 memory used by the initramfs can be released.
 227
 228 At shutdown time,
 229 .I mdmon
 230 should not be killed along with other processes.  Also as it holds a
 231 file (socket actually) open in
 232 .B /dev
 233 (by default) it will not be possible to unmount
 234 .B /dev
 235 if it is a separate filesystem.
 236
 237 .SH EXAMPLES
 238
 239 .B "  mdmon \-\-all-active-arrays \-\-takeover"
 240 .br
 241 Any
 242 .I mdmon
 243 which is currently running is killed and a new instance is started.
 244 This should be run during in the boot sequence if an initramfs was
 245 used, so that any mdmon running from the initramfs will not hold
 246 the initramfs active.
 247 .SH SEE ALSO
 248 .IR mdadm (8),
 249 .IR md (4).