pipe.7: Document /proc files controlling memory usage by pipes

[thirdparty/man-pages.git] / man7 / pipe.7

.\" Copyright (C) 2005 Michael Kerrisk <mtk.manpages@gmail.com>
.\"
.\" %%%LICENSE_START(VERBATIM)
.\" Permission is granted to make and distribute verbatim copies of this
.\" manual provided the copyright notice and this permission notice are
.\" preserved on all copies.
.\"
.\" Permission is granted to copy and distribute modified versions of this
.\" manual under the conditions for verbatim copying, provided that the
.\" entire resulting derived work is distributed under the terms of a
.\" permission notice identical to this one.
.\"
.\" Since the Linux kernel and libraries are constantly changing, this
.\" manual page may be incorrect or out-of-date.  The author(s) assume no
.\" responsibility for errors or omissions, or for damages resulting from
.\" the use of the information contained herein.  The author(s) may not
.\" have taken the same level of care in the production of this manual,
.\" which is licensed free of charge, as they might when working
.\" professionally.
.\"
.\" Formatted or processed versions of this manual, if unaccompanied by
.\" the source, must acknowledge the copyright and authors of this work.
.\" %%%LICENSE_END
.\"
.TH PIPE 7 2016-10-08 "Linux" "Linux Programmer's Manual"
.SH NAME
pipe \- overview of pipes and FIFOs
.SH DESCRIPTION
Pipes and FIFOs (also known as named pipes)
provide a unidirectional interprocess communication channel.
A pipe has a
.I read end
and a
.IR "write end" .
Data written to the write end of a pipe can be read
from the read end of the pipe.

A pipe is created using
.BR pipe (2),
which creates a new pipe and returns two file descriptors,
one referring to the read end of the pipe,
the other referring to the write end.
Pipes can be used to create a communication channel between related
processes; see
.BR pipe (2)
for an example.

A FIFO (short for First In First Out) has a name within the filesystem
(created using
.BR mkfifo (3)),
and is opened using
.BR open (2).
Any process may open a FIFO, assuming the file permissions allow it.
The read end is opened using the
.B O_RDONLY
flag; the write end is opened using the
.B O_WRONLY
flag.
See
.BR fifo (7)
for further details.
.IR Note :
although FIFOs have a pathname in the filesystem,
I/O on FIFOs does not involve operations on the underlying device
(if there is one).
.SS I/O on pipes and FIFOs
The only difference between pipes and FIFOs is the manner in which
they are created and opened.
Once these tasks have been accomplished,
I/O on pipes and FIFOs has exactly the same semantics.

If a process attempts to read from an empty pipe, then
.BR read (2)
will block until data is available.
If a process attempts to write to a full pipe (see below), then
.BR write (2)
blocks until sufficient data has been read from the pipe
to allow the write to complete.
Nonblocking I/O is possible by using the
.BR fcntl (2)
.B F_SETFL
operation to enable the
.B O_NONBLOCK
open file status flag.

The communication channel provided by a pipe is a
.IR "byte stream" :
there is no concept of message boundaries.

If all file descriptors referring to the write end of a pipe
have been closed, then an attempt to
.BR read (2)
from the pipe will see end-of-file
.RB ( read (2)
will return 0).
If all file descriptors referring to the read end of a pipe
have been closed, then a
.BR write (2)
will cause a
.B SIGPIPE
signal to be generated for the calling process.
If the calling process is ignoring this signal, then
.BR write (2)
fails with the error
.BR EPIPE .
An application that uses
.BR pipe (2)
and
.BR fork (2)
should use suitable
.BR close (2)
calls to close unnecessary duplicate file descriptors;
this ensures that end-of-file and
.BR SIGPIPE / EPIPE
are delivered when appropriate.

It is not possible to apply
.BR lseek (2)
to a pipe.
.SS Pipe capacity
A pipe has a limited capacity.
If the pipe is full, then a
.BR write (2)
will block or fail, depending on whether the
.B O_NONBLOCK
flag is set (see below).
Different implementations have different limits for the pipe capacity.
Applications should not rely on a particular capacity:
an application should be designed so that a reading process consumes data
as soon as it is available,
so that a writing process does not remain blocked.

In Linux versions before 2.6.11, the capacity of a pipe was the same as
the system page size (e.g., 4096 bytes on i386).
Since Linux 2.6.11, the pipe capacity is 65536 bytes.
Since Linux 2.6.35, the default pipe capacity is 65536 bytes,
but the capacity can be queried and set using the
.BR fcntl (2)
.BR F_GETPIPE_SZ
and
.BR F_SETPIPE_SZ
operations.
See
.BR fcntl (2)
for more information.

The following
.BR ioctl (2)
operation, which can be applied to a file descriptor
that refers to either end of a pipe,
places a count of the number of unread bytes in the pipe in the
.I int
buffer pointed to by the final argument of the call:

    ioctl(fd, FIONREAD, &nbytes);

The
.B FIONREAD
operation is not specified in any standard,
but is provided on many implementations.
.\"
.SS /proc files
On Linux, the following files control how much memory can be used for pipes:
.TP
.IR /proc/sys/fs/pipe-max-pages " (since Linux 2.6.34)"
.\" commit b492e95be0ae672922f4734acf3f5d35c30be948
Deprecated since Linux 2.6.35.
.TP
.IR /proc/sys/fs/pipe-max-size " (since Linux 2.6.35)"
.\" commit ff9da691c0498ff81fdd014e7a0731dab2337dac
The maximum size (in bytes) of individual pipes created or set by users
without the
.B CAP_SYS_RESOURCE
capability.
.TP
.IR /proc/sys/fs/pipe-user-pages-hard " (since Linux 4.4)"
.\" commit 759c01142a5d0f364a462346168a56de28a80f52
The combined maximum size (in pages) of all pipes created or set by a
single user without both the
.B CAP_SYS_RESOURCE
and
.B CAP_SYS_ADMIN
capabilities.
.TP
.IR /proc/sys/fs/pipe-user-pages-soft " (since Linux 4.4)"
.\" commit 759c01142a5d0f364a462346168a56de28a80f52
The combined size (in pages) of all pipes created or set by a single
user (without both the
.B CAP_SYS_RESOURCE
and
.B CAP_SYS_ADMIN
capabilities) after which individual pipes will be limited to 1 page.
.\"
.SS PIPE_BUF
POSIX.1 says that
.BR write (2)s
of less than
.B PIPE_BUF
bytes must be atomic: the output data is written to the pipe as a
contiguous sequence.
Writes of more than
.B PIPE_BUF
bytes may be nonatomic: the kernel may interleave the data
with data written by other processes.
POSIX.1 requires
.B PIPE_BUF
to be at least 512 bytes.
(On Linux,
.B PIPE_BUF
is 4096 bytes.)
The precise semantics depend on whether the file descriptor is nonblocking
.RB ( O_NONBLOCK ),
whether there are multiple writers to the pipe, and on
.IR n ,
the number of bytes to be written:
.TP
\fBO_NONBLOCK\fP disabled, \fIn\fP <= \fBPIPE_BUF\fP
All
.I n
bytes are written atomically;
.BR write (2)
may block if there is not room for
.I n
bytes to be written immediately
.TP
\fBO_NONBLOCK\fP enabled, \fIn\fP <= \fBPIPE_BUF\fP
If there is room to write
.I n
bytes to the pipe, then
.BR write (2)
succeeds immediately, writing all
.I n
bytes; otherwise
.BR write (2)
fails, with
.I errno
set to
.BR EAGAIN .
.TP
\fBO_NONBLOCK\fP disabled, \fIn\fP > \fBPIPE_BUF\fP
The write is nonatomic: the data given to
.BR write (2)
may be interleaved with
.BR write (2)s
by other process;
the
.BR write (2)
blocks until
.I n
bytes have been written.
.TP
\fBO_NONBLOCK\fP enabled, \fIn\fP > \fBPIPE_BUF\fP
If the pipe is full, then
.BR write (2)
fails, with
.I errno
set to
.BR EAGAIN .
Otherwise, from 1 to
.I n
bytes may be written (i.e., a "partial write" may occur;
the caller should check the return value from
.BR write (2)
to see how many bytes were actually written),
and these bytes may be interleaved with writes by other processes.
.SS Open file status flags
The only open file status flags that can be meaningfully applied to
a pipe or FIFO are
.B O_NONBLOCK
and
.BR O_ASYNC .

Setting the
.B O_ASYNC
flag for the read end of a pipe causes a signal
.RB ( SIGIO
by default) to be generated when new input becomes available on the pipe.
The target for delivery of signals must be set using the
.BR fcntl (2)
.B F_SETOWN
command.
On Linux,
.B O_ASYNC
is supported for pipes and FIFOs only since kernel 2.6.
.SS Portability notes
On some systems (but not Linux), pipes are bidirectional:
data can be transmitted in both directions between the pipe ends.
POSIX.1 requires only unidirectional pipes.
Portable applications should avoid reliance on
bidirectional pipe semantics.
.SH SEE ALSO
.BR mkfifo (1),
.BR dup (2),
.BR fcntl (2),
.BR open (2),
.BR pipe (2),
.BR poll (2),
.BR select (2),
.BR socketpair (2),
.BR splice (2),
.BR stat (2),
.BR mkfifo (3),
.BR epoll (7),
.BR fifo (7)