getent.1, _syscall.2, acct.2, adjtimex.2, bdflush.2, brk.2, cacheflush.2, getsid...

[thirdparty/man-pages.git] / man7 / cpuset.7

.\" Copyright (c) 2008 Silicon Graphics, Inc.
.\"
.\" Author: Paul Jackson (http://oss.sgi.com/projects/cpusets)
.\"
.\" This is free documentation; you can redistribute it and/or
.\" modify it under the terms of the GNU General Public License
.\" version 2 as published by the Free Software Foundation.
.\"
.\" The GNU General Public License's references to "object code"
.\" and "executables" are to be interpreted as the output of any
.\" document formatting or typesetting system, including
.\" intermediate and printed output.
.\"
.\" This manual is distributed in the hope that it will be useful,
.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
.\" GNU General Public License for more details.
.\"
.\" You should have received a copy of the GNU General Public
.\" License along with this manual; if not, see
.\" <http://www.gnu.org/licenses/>.
.\"
.TH CPUSET 7 2013-02-12 "Linux" "Linux Programmer's Manual"
.SH NAME
cpuset \- confine processes to processor and memory node subsets
.SH DESCRIPTION
The cpuset file system is a pseudo-file-system interface
to the kernel cpuset mechanism,
which is used to control the processor placement
and memory placement of processes.
It is commonly mounted at
.IR /dev/cpuset .
.PP
On systems with kernels compiled with built in support for cpusets,
all processes are attached to a cpuset, and cpusets are always present.
If a system supports cpusets, then it will have the entry
.B nodev cpuset
in the file
.IR /proc/filesystems .
By mounting the cpuset file system (see the
.B EXAMPLE
section below),
the administrator can configure the cpusets on a system
to control the processor and memory placement of processes
on that system.
By default, if the cpuset configuration
on a system is not modified or if the cpuset file
system is not even mounted, then the cpuset mechanism,
though present, has no affect on the system's behavior.
.PP
A cpuset defines a list of CPUs and memory nodes.
.PP
The CPUs of a system include all the logical processing
units on which a process can execute, including, if present,
multiple processor cores within a package and Hyper-Threads
within a processor core.
Memory nodes include all distinct
banks of main memory; small and SMP systems typically have
just one memory node that contains all the system's main memory,
while NUMA (non-uniform memory access) systems have multiple memory nodes.
.PP
Cpusets are represented as directories in a hierarchical
pseudo-file system, where the top directory in the hierarchy
.RI ( /dev/cpuset )
represents the entire system (all online CPUs and memory nodes)
and any cpuset that is the child (descendant) of
another parent cpuset contains a subset of that parent's
CPUs and memory nodes.
The directories and files representing cpusets have normal
file-system permissions.
.PP
Every process in the system belongs to exactly one cpuset.
A process is confined to only run on the CPUs in
the cpuset it belongs to, and to allocate memory only
on the memory nodes in that cpuset.
When a process
.BR fork (2)s,
the child process is placed in the same cpuset as its parent.
With sufficient privilege, a process may be moved from one
cpuset to another and the allowed CPUs and memory nodes
of an existing cpuset may be changed.
.PP
When the system begins booting, a single cpuset is
defined that includes all CPUs and memory nodes on the
system, and all processes are in that cpuset.
During the boot process, or later during normal system operation,
other cpusets may be created, as subdirectories of this top cpuset,
under the control of the system administrator,
and processes may be placed in these other cpusets.
.PP
Cpusets are integrated with the
.BR sched_setaffinity (2)
scheduling affinity mechanism and the
.BR mbind (2)
and
.BR set_mempolicy (2)
memory-placement mechanisms in the kernel.
Neither of these mechanisms let a process make use
of a CPU or memory node that is not allowed by that process's cpuset.
If changes to a process's cpuset placement conflict with these
other mechanisms, then cpuset placement is enforced
even if it means overriding these other mechanisms.
The kernel accomplishes this overriding by silently
restricting the CPUs and memory nodes requested by
these other mechanisms to those allowed by the
invoking process's cpuset.
This can result in these
other calls returning an error, if for example, such
a call ends up requesting an empty set of CPUs or
memory nodes, after that request is restricted to
the invoking process's cpuset.
.PP
Typically, a cpuset is used to manage
the CPU and memory-node confinement for a set of
cooperating processes such as a batch scheduler job, and these
other mechanisms are used to manage the placement of
individual processes or memory regions within that set or job.
.SH FILES
Each directory below
.I /dev/cpuset
represents a cpuset and contains a fixed set of pseudo-files
describing the state of that cpuset.
.PP
New cpusets are created using the
.BR mkdir (2)
system call or the
.BR mkdir (1)
command.
The properties of a cpuset, such as its flags, allowed
CPUs and memory nodes, and attached processes, are queried and modified
by reading or writing to the appropriate file in that cpuset's directory,
as listed below.
.PP
The pseudo-files in each cpuset directory are automatically created when
the cpuset is created, as a result of the
.BR mkdir (2)
invocation.
It is not possible to directly add or remove these pseudo-files.
.PP
A cpuset directory that contains no child cpuset directories,
and has no attached processes, can be removed using
.BR rmdir (2)
or
.BR rmdir (1).
It is not necessary, or possible,
to remove the pseudo-files inside the directory before removing it.
.PP
The pseudo-files in each cpuset directory are
small text files that may be read and
written using traditional shell utilities such as
.BR cat (1),
and
.BR echo (1),
or from a program by using file I/O library functions or system calls,
such as
.BR open (2),
.BR read (2),
.BR write (2),
and
.BR close (2).
.PP
The pseudo-files in a cpuset directory represent internal kernel
state and do not have any persistent image on disk.
Each of these per-cpuset files is listed and described below.
.\" ====================== tasks ======================
.TP
.I tasks
List of the process IDs (PIDs) of the processes in that cpuset.
The list is formatted as a series of ASCII
decimal numbers, each followed by a newline.
A process may be added to a cpuset (automatically removing
it from the cpuset that previously contained it) by writing its
PID to that cpuset's
.I tasks
file (with or without a trailing newline.)

.B Warning:
only one PID may be written to the
.I tasks
file at a time.
If a string is written that contains more
than one PID, only the first one will be used.
.\" =================== notify_on_release ===================
.TP
.I notify_on_release
Flag (0 or 1).
If set (1), that cpuset will receive special handling
after it is released, that is, after all processes cease using
it (i.e., terminate or are moved to a different cpuset)
and all child cpuset directories have been removed.
See the \fBNotify On Release\fR section, below.
.\" ====================== cpus ======================
.TP
.I cpuset.cpus
List of the physical numbers of the CPUs on which processes
in that cpuset are allowed to execute.
See \fBList Format\fR below for a description of the
format of
.IR cpus .

The CPUs allowed to a cpuset may be changed by
writing a new list to its
.I cpus
file.
.\" ==================== cpu_exclusive ====================
.TP
.I cpuset.cpu_exclusive
Flag (0 or 1).
If set (1), the cpuset has exclusive use of
its CPUs (no sibling or cousin cpuset may overlap CPUs).
By default this is off (0).
Newly created cpusets also initially default this to off (0).

Two cpusets are
.I sibling
cpusets if they share the same parent cpuset in the
.I /dev/cpuset
hierarchy.
Two cpusets are
.I cousin
cpusets if neither is the ancestor of the other.
Regardless of the
.I cpu_exclusive
setting, if one cpuset is the ancestor of another,
and if both of these cpusets have nonempty
.IR cpus ,
then their
.I cpus
must overlap, because the
.I cpus
of any cpuset are always a subset of the
.I cpus
of its parent cpuset.
.\" ====================== mems ======================
.TP
.I cpuset.mems
List of memory nodes on which processes in this cpuset are
allowed to allocate memory.
See \fBList Format\fR below for a description of the
format of
.IR mems .
.\" ==================== mem_exclusive ====================
.TP
.I cpuset.mem_exclusive
Flag (0 or 1).
If set (1), the cpuset has exclusive use of
its memory nodes (no sibling or cousin may overlap).
Also if set (1), the cpuset is a \fBHardwall\fR cpuset (see below.)
By default this is off (0).
Newly created cpusets also initially default this to off (0).

Regardless of the
.I mem_exclusive
setting, if one cpuset is the ancestor of another,
then their memory nodes must overlap, because the memory