man2/mbind.2

   1 .\" Copyright 2003,2004 Andi Kleen, SuSE Labs.
   2 .\" and Copyright 2007 Lee Schermerhorn, Hewlett Packard
   3 .\"
   4 .\" Permission is granted to make and distribute verbatim copies of this
   5 .\" manual provided the copyright notice and this permission notice are
   6 .\" preserved on all copies.
   7 .\"
   8 .\" Permission is granted to copy and distribute modified versions of this
   9 .\" manual under the conditions for verbatim copying, provided that the
  10 .\" entire resulting derived work is distributed under the terms of a
  11 .\" permission notice identical to this one.
  12 .\"
  13 .\" Since the Linux kernel and libraries are constantly changing, this
  14 .\" manual page may be incorrect or out-of-date.  The author(s) assume no
  15 .\" responsibility for errors or omissions, or for damages resulting from
  16 .\" the use of the information contained herein.
  17 .\"
  18 .\" Formatted or processed versions of this manual, if unaccompanied by
  19 .\" the source, must acknowledge the copyright and authors of this work.
  20 .\"
  21 .\" 2006-02-03, mtk, substantial wording changes and other improvements
  22 .\" 2007-08-27, Lee Schermerhorn <Lee.Schermerhorn@hp.com>
  23 .\"     more precise specification of behavior.
  24 .\"
  25 .TH MBIND 2 2008-08-11 Linux "Linux Programmer's Manual"
  26 .SH NAME
  27 mbind \- Set memory policy for a memory range
  28 .SH SYNOPSIS
  29 .nf
  30 .B "#include <numaif.h>"
  31 .sp
  32 .BI "int mbind(void *" addr ", unsigned long " len  ", int " mode ,
  33 .BI "          unsigned long *" nodemask  ", unsigned long " maxnode ,
  34 .BI "          unsigned " flags );
  35 .sp
  36 Link with \fI\-lnuma\fP
  37 .fi
  38 .SH DESCRIPTION
  39 .BR mbind ()
  40 sets the NUMA memory policy,
  41 which consists of a policy mode and zero or more nodes,
  42 for the memory range starting with
  43 .I addr
  44 and continuing for
  45 .I len
  46 bytes.
  47 The memory of a NUMA machine is divided into multiple nodes.
  48 The memory policy defines from which node memory is allocated.
  49
  50 If the memory range specified by the
  51 .IR addr " and " len
  52 arguments includes an "anonymous" region of memory\(emthat is
  53 a region of memory created using the
  54 .BR mmap (2)
  55 system call with the
  56 .BR MAP_ANONYMOUS \(emor
  57 a memory mapped file, mapped using the
  58 .BR mmap (2)
  59 system call with the
  60 .B MAP_PRIVATE
  61 flag, pages will only be allocated according to the specified
  62 policy when the application writes [stores] to the page.
  63 For anonymous regions, an initial read access will use a shared
  64 page in the kernel containing all zeros.
  65 For a file mapped with
  66 .BR MAP_PRIVATE ,
  67 an initial read access will allocate pages according to the
  68 process policy of the process that causes the page to be allocated.
  69 This may not be the process that called
  70 .BR mbind ().
  71
  72 The specified policy will be ignored for any
  73 .B MAP_SHARED
  74 mappings in the specified memory range.
  75 Rather the pages will be allocated according to the process policy
  76 of the process that caused the page to be allocated.
  77 Again, this may not be the process that called
  78 .BR mbind ().
  79
  80 If the specified memory range includes a shared memory region
  81 created using the
  82 .BR shmget (2)
  83 system call and attached using the
  84 .BR shmat (2)
  85 system call,
  86 pages allocated for the anonymous or shared memory region will
  87 be allocated according to the policy specified, regardless which
  88 process attached to the shared memory segment causes the allocation.
  89 If, however, the shared memory region was created with the
  90 .B SHM_HUGETLB
  91 flag,
  92 the huge pages will be allocated according to the policy specified
  93 only if the page allocation is caused by the process that calls
  94 .BR mbind ()
  95 for that region.
  96
  97 By default,
  98 .BR mbind ()
  99 only has an effect for new allocations; if the pages inside
 100 the range have been already touched before setting the policy,
 101 then the policy has no effect.
 102 This default behavior may be overridden by the
 103 .B MPOL_MF_MOVE
 104 and
 105 .B MPOL_MF_MOVE_ALL
 106 flags described below.
 107
 108 The
 109 .I mode
 110 argument must specify one of
 111 .BR MPOL_DEFAULT ,
 112 .BR MPOL_BIND ,
 113 .B MPOL_INTERLEAVE
 114 or
 115 .BR MPOL_PREFERRED .
 116 All policy modes except
 117 .B MPOL_DEFAULT
 118 require the caller to specify via the
 119 .I nodemask
 120 argument,
 121 the node or nodes to which the mode applies.
 122
 123 The
 124 .I mode
 125 argument may also include an optional
 126 .IR "mode flag ".
 127 The supported
 128 .I "mode flags"
 129 are:
 130 .TP
 131 .BR MPOL_F_STATIC_NODES " (since Linux-2.6.26)"
 132 A non-empty
 133 .I nodemask
 134 specifies physical node ids.
 135 Linux does will not remap the
 136 .I nodemask
 137 when the process moves to a different cpuset context,
 138 nor when the set of nodes allowed by the process's
 139 current cpuset context changes.
 140 .TP
 141 .BR MPOL_F_RELATIVE_NODES " (since Linux-2.6.26)"
 142 A non-empty
 143 .I nodemask
 144 specifies node ids that are relative to the set of
 145 node ids allowed  by the process's current cpuset.
 146 .PP
 147 .I nodemask
 148 points to a bitmask of nodes containing up to
 149 .I maxnode
 150 bits.
 151 The bit mask size is rounded to the next multiple of
 152 .IR "sizeof(unsigned long)" ,
 153 but the kernel will only use bits up to
 154 .IR maxnode .
 155 A NULL value of
 156 .I nodemask
 157 or a
 158 .I maxnode
 159 value of zero specifies the empty set of nodes.
 160 If the value of
 161 .I maxnode
 162 is zero,
 163 the
 164 .I nodemask
 165 argument is ignored.
 166 Where a
 167 .I nodemask
 168 is required, it must contain at least one node that is on-line,
 169 allowed by the process's current cpuset context
 170 [unless the
 171 .B MPOL_F_STATIC_NODES
 172 mode flag is specified],
 173 and contains memory.
 174
 175 The
 176 .B MPOL_DEFAULT
 177 mode requests that any non-default policy be removed,
 178 restoring default behavior.
 179 When applied to a range of memory via
 180 .BR mbind (),
 181 this means to use the process policy,
 182 which may have been set with
 183 .BR set_mempolicy (2).
 184 If the mode of the process policy is also
 185 .BR MPOL_DEFAULT ,
 186 the system-wide default policy will be used.
 187 The system-wide default policy allocates
 188 pages on the node of the CPU that triggers the allocation.
 189 For
 190 .BR MPOL_DEFAULT ,
 191 the
 192 .I nodemask
 193 and
 194 .I maxnode
 195 arguments must be specify the empty set of nodes.
 196
 197 The
 198 .B MPOL_BIND
 199 mode specifies a strict policy that restricts memory allocation to
 200 the nodes specified in
 201 .IR nodemask .
 202 If
 203 .I nodemask
 204 specifies more than one node, page allocations will come from
 205 the node with the lowest numeric node ID first, until that node
 206 contains no free memory.
 207 Allocations will then come from the node with the next highest
 208 node ID specified in
 209 .I nodemask
 210 and so forth, until none of the specified nodes contain free memory.
 211 Pages will not be allocated from any node not specified in the
 212 .IR nodemask .
 213
 214 The
 215 .B MPOL_INTERLEAVE
 216 mode specifies that page allocations be interleaved across the
 217 set of nodes specified in
 218 .IR nodemask .
 219 This optimizes for bandwidth instead of latency
 220 by spreading out pages and memory accesses to those pages across
 221 multiple nodes.
 222 To be effective the memory area should be fairly large,
 223 at least 1MB or bigger with a fairly uniform access pattern.
 224 Accesses to a single page of the area will still be limited to
 225 the memory bandwidth of a single node.
 226
 227 .B MPOL_PREFERRED
 228 sets the preferred node for allocation.
 229 The kernel will try to allocate pages from this
 230 node first and fall back to other nodes if the
 231 preferred nodes is low on free memory.
 232 If
 233 .I nodemask
 234 specifies more than one node ID, the first node in the
 235 mask will be selected as the preferred node.
 236 If the
 237 .I nodemask
 238 and
 239 .I maxnode
 240 arguments specify the empty set, then the memory is allocated on
 241 the node of the CPU that triggered the allocation.
 242 This is the only way to specify "local allocation" for a
 243 range of memory via
 244 .BR mbind ().
 245
 246 If
 247 .B MPOL_MF_STRICT
 248 is passed in
 249 .I flags
 250 and
 251 .I policy
 252 is not
 253 .BR MPOL_DEFAULT ,
 254 then the call will fail with the error
 255 .B EIO
 256 if the existing pages in the memory range don't follow the policy.
 257 .\" According to the kernel code, the following is not true
 258 .\" --Lee Schermerhorn
 259 .\" In 2.6.16 or later the kernel will also try to move pages
 260 .\" to the requested node with this flag.
 261
 262 If
 263 .B MPOL_MF_MOVE
 264 is specified in
 265 .IR flags ,
 266 then the kernel will attempt to move all the existing pages
 267 in the memory range so that they follow the policy.
 268 Pages that are shared with other processes will not be moved.
 269 If
 270 .B MPOL_MF_STRICT
 271 is also specified, then the call will fail with the error
 272 .B EIO
 273 if some pages could not be moved.
 274
 275 If
 276 .B MPOL_MF_MOVE_ALL
 277 is passed in
 278 .IR flags ,
 279 then the kernel will attempt to move all existing pages in the memory range
 280 regardless of whether other processes use the pages.
 281 The calling process must be privileged
 282 .RB ( CAP_SYS_NICE )
 283 to use this flag.
 284 If
 285 .B MPOL_MF_STRICT
 286 is also specified, then the call will fail with the error
 287 .B EIO
 288 if some pages could not be moved.
 289 .\" ---------------------------------------------------------------
 290 .SH RETURN VALUE
 291 On success,
 292 .BR mbind ()
 293 returns 0;
 294 on error, \-1 is returned and
 295 .I errno
 296 is set to indicate the error.
 297 .\" ---------------------------------------------------------------
 298 .SH ERRORS
 299 .\"  I think I got all of the error returns.  --Lee Schermerhorn
 300 .TP
 301 .B EFAULT
 302 Part of all of the memory range specified by
 303 .I nodemask
 304 and
 305 .I maxnode
 306 points outside your accessible address space.
 307 Or, there was an unmapped hole in the specified memory range.
 308 .TP
 309 .B EINVAL
 310 An invalid value was specified for
 311 .I flags
 312 or
 313 .IR mode ;
 314 or
 315 .I addr + len
 316 was less than
 317 .IR addr ;
 318 or
 319 .I addr
 320 is not a multiple of the system page size.
 321 Or,
 322 .I mode
 323 is
 324 .B MPOL_DEFAULT
 325 and
 326 .I nodemask
 327 specified a non-empty set;
 328 or
 329 .I mode
 330 is
 331 .B MPOL_BIND
 332 or
 333 .B MPOL_INTERLEAVE
 334 and
 335 .I nodemask
 336 is empty.
 337 Or,
 338 .I maxnode
 339 exceeds a kernel-imposed limit.
 340 .\" As at 2.6.23, this limit is "a page worth of bits", e.g.,
 341 .\" 8 * 4096 bits, assuming a 4kB page size.
 342 Or,
 343 .I nodemask
 344 specifies one or more node IDs that are
 345 greater than the maximum supported node ID.
 346 Or, none of the node IDs specified by
 347 .I nodemask
 348 are on-line and allowed by the process's current cpuset context,
 349 or none of the specified nodes contain memory.
 350 Or, the
 351 .I mode
 352 argument specified both
 353 .B MPOL_F_STATIC_NODES
 354 and
 355 .BR MPOL_F_RELATIVE_NODES .
 356 .TP
 357 .B EIO
 358 .B MPOL_MF_STRICT
 359 was specified and an existing page was already on a node
 360 that does not follow the policy;
 361 or
 362 .B MPOL_MF_MOVE
 363 or
 364 .B MPOL_MF_MOVE_ALL
 365 was specified and the kernel was unable to move all existing
 366 pages in the range.
 367 .TP
 368 .B ENOMEM
 369 Insufficient kernel memory was available.
 370 .TP
 371 .B EPERM
 372 The
 373 .I flags
 374 argument included the
 375 .B MPOL_MF_MOVE_ALL
 376 flag and the caller does not have the
 377 .B CAP_SYS_NICE
 378 privilege.
 379 .\" ---------------------------------------------------------------
 380 .SH CONFORMING TO
 381 This system call is Linux-specific.
 382 .SH NOTES
 383 NUMA policy is not supported on a memory mapped file range
 384 that was mapped with the
 385 .B MAP_SHARED
 386 flag.
 387
 388 .B MPOL_MF_STRICT
 389 is ignored on huge page mappings.
 390
 391 The
 392 .BR MPOL_DEFAULT ,
 393 mode can have different effects for
 394 .BR mbind ()
 395 and
 396 .BR set_mempolicy (2).
 397 When
 398 .B MPOL_DEFAULT
 399 is specified for
 400 .BR set_mempolicy (2),
 401 the process's policy reverts to system default policy
 402 or local allocation.
 403 When
 404 .B MPOL_DEFAULT
 405 is specified for a range of memory using
 406 .BR mbind (),
 407 any pages subsequently allocated for that range will use
 408 the process's policy, as set by
 409 .BR set_mempolicy (2).
 410 This effectively removes the explicit policy from the
 411 specified range, "falling back" to a possibly non-default
 412 policy.
 413 To select explicit "local allocation" for a memory range,
 414 specify a
 415 .I mode
 416 of
 417 .B MPOL_PREFERRED
 418 with an empty set of nodes.
 419 This method will work for
 420 .BR set_mempolicy (2),
 421 as well.
 422 .\" ---------------------------------------------------------------
 423 .SS "Versions and Library Support"
 424 The
 425 .BR mbind (),
 426 .BR get_mempolicy (2),
 427 and
 428 .BR set_mempolicy (2)
 429 system calls were added to the Linux kernel with version 2.6.7.
 430 They are only available on kernels compiled with
 431 .BR CONFIG_NUMA .
 432
 433 You can link with
 434 .I \-lnuma
 435 to get system call definitions.
 436 .I libnuma
 437 and the required
 438 .I <numaif.h>
 439 header are available in the
 440 .I numactl
 441 package.
 442
 443 However, applications should not use these system calls directly.
 444 Instead, the higher level interface provided by the
 445 .BR numa (3)
 446 functions in the
 447 .I numactl
 448 package is recommended.
 449 The
 450 .I numactl
 451 package is available at
 452 .IR ftp://oss.sgi.com/www/projects/libnuma/download/ .
 453 The package is also included in some Linux distributions.
 454 Some distributions include the development library and header
 455 in the separate
 456 .I numactl-devel
 457 package.
 458
 459 Support for huge page policy was added with 2.6.16.
 460 For interleave policy to be effective on huge page mappings the
 461 policied memory needs to be tens of megabytes or larger.
 462
 463 .B MPOL_MF_MOVE
 464 and
 465 .B MPOL_MF_MOVE_ALL
 466 are only available on Linux 2.6.16 and later.
 467 .SH SEE ALSO
 468 .BR get_mempolicy (2),
 469 .BR getcpu (2),
 470 .BR mmap (2),
 471 .BR set_mempolicy (2),
 472 .BR shmat (2),
 473 .BR shmget (2),
 474 .BR numa (3),
 475 .BR cpuset (7),
 476 .BR numactl (8)