]>
Commit | Line | Data |
---|---|---|
c11b1abf | 1 | .\" Copyright (C) 2005 Michael Kerrisk <mtk.manpages@gmail.com> |
2adb3bd6 | 2 | .\" |
93015253 | 3 | .\" %%%LICENSE_START(VERBATIM) |
2adb3bd6 MK |
4 | .\" Permission is granted to make and distribute verbatim copies of this |
5 | .\" manual provided the copyright notice and this permission notice are | |
6 | .\" preserved on all copies. | |
7 | .\" | |
8 | .\" Permission is granted to copy and distribute modified versions of this | |
9 | .\" manual under the conditions for verbatim copying, provided that the | |
10 | .\" entire resulting derived work is distributed under the terms of a | |
11 | .\" permission notice identical to this one. | |
c13182ef | 12 | .\" |
2adb3bd6 MK |
13 | .\" Since the Linux kernel and libraries are constantly changing, this |
14 | .\" manual page may be incorrect or out-of-date. The author(s) assume no | |
15 | .\" responsibility for errors or omissions, or for damages resulting from | |
10d76543 MK |
16 | .\" the use of the information contained herein. The author(s) may not |
17 | .\" have taken the same level of care in the production of this manual, | |
18 | .\" which is licensed free of charge, as they might when working | |
19 | .\" professionally. | |
c13182ef | 20 | .\" |
2adb3bd6 MK |
21 | .\" Formatted or processed versions of this manual, if unaccompanied by |
22 | .\" the source, must acknowledge the copyright and authors of this work. | |
4b72fb64 | 23 | .\" %%%LICENSE_END |
2adb3bd6 | 24 | .\" |
b8efb414 | 25 | .TH PIPE 7 2016-10-08 "Linux" "Linux Programmer's Manual" |
2adb3bd6 | 26 | .SH NAME |
3a20b4ca | 27 | pipe \- overview of pipes and FIFOs |
2adb3bd6 | 28 | .SH DESCRIPTION |
c13182ef | 29 | Pipes and FIFOs (also known as named pipes) |
2adb3bd6 | 30 | provide a unidirectional interprocess communication channel. |
c13182ef MK |
31 | A pipe has a |
32 | .I read end | |
33 | and a | |
2adb3bd6 MK |
34 | .IR "write end" . |
35 | Data written to the write end of a pipe can be read | |
48afe71d | 36 | from the read end of the pipe. |
2adb3bd6 | 37 | |
c13182ef | 38 | A pipe is created using |
2adb3bd6 MK |
39 | .BR pipe (2), |
40 | which creates a new pipe and returns two file descriptors, | |
c13182ef | 41 | one referring to the read end of the pipe, |
2adb3bd6 | 42 | the other referring to the write end. |
c13182ef MK |
43 | Pipes can be used to create a communication channel between related |
44 | processes; see | |
6d563954 MK |
45 | .BR pipe (2) |
46 | for an example. | |
2adb3bd6 | 47 | |
28955f15 SP |
48 | A FIFO (short for First In First Out) has a name within the filesystem |
49 | (created using | |
2adb3bd6 MK |
50 | .BR mkfifo (3)), |
51 | and is opened using | |
c13182ef | 52 | .BR open (2). |
2adb3bd6 | 53 | Any process may open a FIFO, assuming the file permissions allow it. |
c13182ef | 54 | The read end is opened using the |
2adb3bd6 MK |
55 | .B O_RDONLY |
56 | flag; the write end is opened using the | |
57 | .B O_WRONLY | |
58 | flag. | |
59 | See | |
af5b2ef2 | 60 | .BR fifo (7) |
2adb3bd6 | 61 | for further details. |
c13182ef | 62 | .IR Note : |
9ee4a2b6 | 63 | although FIFOs have a pathname in the filesystem, |
c13182ef | 64 | I/O on FIFOs does not involve operations on the underlying device |
48afe71d | 65 | (if there is one). |
73d8cece | 66 | .SS I/O on pipes and FIFOs |
2adb3bd6 | 67 | The only difference between pipes and FIFOs is the manner in which |
c13182ef MK |
68 | they are created and opened. |
69 | Once these tasks have been accomplished, | |
2adb3bd6 | 70 | I/O on pipes and FIFOs has exactly the same semantics. |
2adb3bd6 | 71 | |
2adb3bd6 | 72 | If a process attempts to read from an empty pipe, then |
c13182ef | 73 | .BR read (2) |
2adb3bd6 MK |
74 | will block until data is available. |
75 | If a process attempts to write to a full pipe (see below), then | |
76 | .BR write (2) | |
77 | blocks until sufficient data has been read from the pipe | |
78 | to allow the write to complete. | |
44732c9c | 79 | Nonblocking I/O is possible by using the |
2adb3bd6 | 80 | .BR fcntl (2) |
c13182ef | 81 | .B F_SETFL |
2adb3bd6 MK |
82 | operation to enable the |
83 | .B O_NONBLOCK | |
84 | open file status flag. | |
85 | ||
48afe71d | 86 | The communication channel provided by a pipe is a |
c13182ef | 87 | .IR "byte stream" : |
48afe71d MK |
88 | there is no concept of message boundaries. |
89 | ||
c13182ef MK |
90 | If all file descriptors referring to the write end of a pipe |
91 | have been closed, then an attempt to | |
2adb3bd6 MK |
92 | .BR read (2) |
93 | from the pipe will see end-of-file | |
94 | .RB ( read (2) | |
95 | will return 0). | |
c13182ef | 96 | If all file descriptors referring to the read end of a pipe |
2adb3bd6 MK |
97 | have been closed, then a |
98 | .BR write (2) | |
99 | will cause a | |
100 | .B SIGPIPE | |
101 | signal to be generated for the calling process. | |
102 | If the calling process is ignoring this signal, then | |
103 | .BR write (2) | |
104 | fails with the error | |
105 | .BR EPIPE . | |
c13182ef | 106 | An application that uses |
2adb3bd6 MK |
107 | .BR pipe (2) |
108 | and | |
109 | .BR fork (2) | |
c13182ef | 110 | should use suitable |
2adb3bd6 MK |
111 | .BR close (2) |
112 | calls to close unnecessary duplicate file descriptors; | |
c13182ef MK |
113 | this ensures that end-of-file and |
114 | .BR SIGPIPE / EPIPE | |
2adb3bd6 MK |
115 | are delivered when appropriate. |
116 | ||
48afe71d | 117 | It is not possible to apply |
2adb3bd6 | 118 | .BR lseek (2) |
48afe71d | 119 | to a pipe. |
73d8cece | 120 | .SS Pipe capacity |
2adb3bd6 MK |
121 | A pipe has a limited capacity. |
122 | If the pipe is full, then a | |
123 | .BR write (2) | |
124 | will block or fail, depending on whether the | |
c13182ef | 125 | .B O_NONBLOCK |
2adb3bd6 MK |
126 | flag is set (see below). |
127 | Different implementations have different limits for the pipe capacity. | |
c13182ef MK |
128 | Applications should not rely on a particular capacity: |
129 | an application should be designed so that a reading process consumes data | |
130 | as soon as it is available, | |
2adb3bd6 MK |
131 | so that a writing process does not remain blocked. |
132 | ||
c13182ef | 133 | In Linux versions before 2.6.11, the capacity of a pipe was the same as |
34ccb744 | 134 | the system page size (e.g., 4096 bytes on i386). |
2adb3bd6 | 135 | Since Linux 2.6.11, the pipe capacity is 65536 bytes. |
640a438d MK |
136 | Since Linux 2.6.35, the default pipe capacity is 65536 bytes, |
137 | but the capacity can be queried and set using the | |
91d4de66 EDB |
138 | .BR fcntl (2) |
139 | .BR F_GETPIPE_SZ | |
140 | and | |
141 | .BR F_SETPIPE_SZ | |
142 | operations. | |
143 | See | |
144 | .BR fcntl (2) | |
145 | for more information. | |
2293a55f MK |
146 | |
147 | The following | |
148 | .BR ioctl (2) | |
149 | operation, which can be applied to a file descriptor | |
150 | that refers to either end of a pipe, | |
151 | places a count of the number of unread bytes in the pipe in the | |
152 | .I int | |
153 | buffer pointed to by the final argument of the call: | |
154 | ||
155 | ioctl(fd, FIONREAD, &nbytes); | |
156 | ||
157 | The | |
158 | .B FIONREAD | |
159 | operation is not specified in any standard, | |
160 | but is provided on many implementations. | |
787dd4ad | 161 | .\" |
81c4577e VN |
162 | .SS /proc files |
163 | On Linux, the following files control how much memory can be used for pipes: | |
164 | .TP | |
12033ac4 | 165 | .IR /proc/sys/fs/pipe-max-pages " (only in Linux 2.6.34)" |
81c4577e | 166 | .\" commit b492e95be0ae672922f4734acf3f5d35c30be948 |
12033ac4 MK |
167 | An upper limit, in pages, on the capacity that an unprivileged user |
168 | (one without the | |
169 | .BR CAP_SYS_RESOURCE | |
170 | capability) | |
171 | can set for a pipe. | |
172 | ||
173 | The default value for this limit is 16 times the default pipe capacity | |
174 | (see above); the lower limit is two pages. | |
175 | ||
176 | This interface was removed in Linux 2.6.35, in favor of | |
177 | .IR /proc/sys/fs/pipe-max-size . | |
81c4577e VN |
178 | .TP |
179 | .IR /proc/sys/fs/pipe-max-size " (since Linux 2.6.35)" | |
180 | .\" commit ff9da691c0498ff81fdd014e7a0731dab2337dac | |
cc6b4da2 MK |
181 | The maximum size (in bytes) of individual pipes that can be set |
182 | .\" This limit is not checked on pipe creation, where the capacity is | |
183 | .\" always PIPE_DEF_BUFS, regardless of pipe-max-size | |
184 | by users without the | |
81c4577e VN |
185 | .B CAP_SYS_RESOURCE |
186 | capability. | |
33dc4b59 MK |
187 | The value assigned to this file may be rounded upward, |
188 | to reflect the value actually employed for a convenient implementation. | |
189 | To determine the rounded-up value, | |
190 | display the contents of this file after assigning a value to it. | |
191 | ||
cc6b4da2 | 192 | The default value for this file is 1048576 (1 MiB). |
33dc4b59 | 193 | The minimum value that can be assigned to this file is the system page size. |
12033ac4 MK |
194 | Attempts to set a limit less than the page size cause |
195 | .BR write (2) | |
196 | to fail with the error | |
197 | .BR EINVAL . | |
81c4577e | 198 | .TP |
12033ac4 | 199 | .IR /proc/sys/fs/pipe-user-pages-hard " (since Linux 4.5)" |
81c4577e | 200 | .\" commit 759c01142a5d0f364a462346168a56de28a80f52 |
12033ac4 MK |
201 | The hard limit on the total size (in pages) of all pipes created or set by |
202 | a single unprivileged user (i.e., one with neither the | |
81c4577e | 203 | .B CAP_SYS_RESOURCE |
12033ac4 | 204 | nor the |
81c4577e | 205 | .B CAP_SYS_ADMIN |
12033ac4 MK |
206 | capability). |
207 | So long as the total number of pages allocated to pipe buffers | |
208 | for this user is at this limit, | |
209 | attempts to create new pipes will be denied, | |
210 | and attempts to increase a pipe's capacity will be denied. | |
211 | ||
212 | When the value of this limit is zero (which is the default), | |
213 | no hard limit is applied. | |
214 | .\" The default was chosen to avoid breaking existing applications that | |
215 | .\" make intensive use of pipes (e.g., for splicing). | |
81c4577e | 216 | .TP |
12033ac4 | 217 | .IR /proc/sys/fs/pipe-user-pages-soft " (since Linux 4.5)" |
81c4577e | 218 | .\" commit 759c01142a5d0f364a462346168a56de28a80f52 |
12033ac4 MK |
219 | The soft limit on the total size (in pages) of all pipes created or set by |
220 | a single unprivileged user (i.e., one with neither the | |
81c4577e | 221 | .B CAP_SYS_RESOURCE |
12033ac4 | 222 | nor the |
81c4577e | 223 | .B CAP_SYS_ADMIN |
12033ac4 MK |
224 | capability). |
225 | So long as the total number of pages allocated to pipe buffers | |
226 | for this user is at this limit, | |
227 | individual pipes created by a user will be limited to one page, | |
228 | and attempts to increase a pipe's capacity will be denied. | |
229 | ||
230 | When the value of this limit is zero, no soft limit is applied. | |
231 | The default value for this file is 16384, | |
232 | which permits creating up to 1024 pipes with the default capacity. | |
1cb4e899 MK |
233 | .PP |
234 | Before Linux 4.9, some bugs affected the handling of the | |
235 | .IR pipe-user-pages-soft | |
236 | and | |
237 | .IR pipe-user-pages-hard | |
238 | limits; see BUGS. | |
81c4577e | 239 | .\" |
2adb3bd6 | 240 | .SS PIPE_BUF |
3330e739 | 241 | POSIX.1 says that |
2adb3bd6 | 242 | .BR write (2)s |
c13182ef MK |
243 | of less than |
244 | .B PIPE_BUF | |
245 | bytes must be atomic: the output data is written to the pipe as a | |
2adb3bd6 MK |
246 | contiguous sequence. |
247 | Writes of more than | |
248 | .B PIPE_BUF | |
24b74457 | 249 | bytes may be nonatomic: the kernel may interleave the data |
2adb3bd6 | 250 | with data written by other processes. |
3330e739 | 251 | POSIX.1 requires |
c13182ef MK |
252 | .B PIPE_BUF |
253 | to be at least 512 bytes. | |
254 | (On Linux, | |
255 | .B PIPE_BUF | |
2adb3bd6 | 256 | is 4096 bytes.) |
ff40dbb3 | 257 | The precise semantics depend on whether the file descriptor is nonblocking |
2adb3bd6 | 258 | .RB ( O_NONBLOCK ), |
c13182ef | 259 | whether there are multiple writers to the pipe, and on |
2adb3bd6 MK |
260 | .IR n , |
261 | the number of bytes to be written: | |
262 | .TP | |
263 | \fBO_NONBLOCK\fP disabled, \fIn\fP <= \fBPIPE_BUF\fP | |
264 | All | |
265 | .I n | |
266 | bytes are written atomically; | |
267 | .BR write (2) | |
268 | may block if there is not room for | |
269 | .I n | |
270 | bytes to be written immediately | |
271 | .TP | |
272 | \fBO_NONBLOCK\fP enabled, \fIn\fP <= \fBPIPE_BUF\fP | |
273 | If there is room to write | |
274 | .I n | |
275 | bytes to the pipe, then | |
276 | .BR write (2) | |
277 | succeeds immediately, writing all | |
278 | .I n | |
c13182ef | 279 | bytes; otherwise |
2adb3bd6 MK |
280 | .BR write (2) |
281 | fails, with | |
282 | .I errno | |
283 | set to | |
284 | .BR EAGAIN . | |
285 | .TP | |
286 | \fBO_NONBLOCK\fP disabled, \fIn\fP > \fBPIPE_BUF\fP | |
24b74457 | 287 | The write is nonatomic: the data given to |
c13182ef MK |
288 | .BR write (2) |
289 | may be interleaved with | |
290 | .BR write (2)s | |
291 | by other process; | |
2adb3bd6 MK |
292 | the |
293 | .BR write (2) | |
c13182ef | 294 | blocks until |
2adb3bd6 MK |
295 | .I n |
296 | bytes have been written. | |
297 | .TP | |
298 | \fBO_NONBLOCK\fP enabled, \fIn\fP > \fBPIPE_BUF\fP | |
c13182ef | 299 | If the pipe is full, then |
2adb3bd6 MK |
300 | .BR write (2) |
301 | fails, with | |
302 | .I errno | |
303 | set to | |
304 | .BR EAGAIN . | |
305 | Otherwise, from 1 to | |
c13182ef | 306 | .I n |
2adb3bd6 | 307 | bytes may be written (i.e., a "partial write" may occur; |
c13182ef | 308 | the caller should check the return value from |
2adb3bd6 MK |
309 | .BR write (2) |
310 | to see how many bytes were actually written), | |
311 | and these bytes may be interleaved with writes by other processes. | |
73d8cece | 312 | .SS Open file status flags |
c13182ef | 313 | The only open file status flags that can be meaningfully applied to |
48afe71d MK |
314 | a pipe or FIFO are |
315 | .B O_NONBLOCK | |
c13182ef | 316 | and |
48afe71d MK |
317 | .BR O_ASYNC . |
318 | ||
319 | Setting the | |
320 | .B O_ASYNC | |
c13182ef | 321 | flag for the read end of a pipe causes a signal |
48afe71d | 322 | .RB ( SIGIO |
98faa645 MK |
323 | by default) to be generated when new input becomes available on the pipe. |
324 | The target for delivery of signals must be set using the | |
48afe71d | 325 | .BR fcntl (2) |
98faa645 MK |
326 | .B F_SETOWN |
327 | command. | |
48afe71d | 328 | On Linux, |
c13182ef | 329 | .B O_ASYNC |
48afe71d | 330 | is supported for pipes and FIFOs only since kernel 2.6. |
73d8cece | 331 | .SS Portability notes |
c13182ef | 332 | On some systems (but not Linux), pipes are bidirectional: |
2adb3bd6 | 333 | data can be transmitted in both directions between the pipe ends. |
a448fdd6 | 334 | POSIX.1 requires only unidirectional pipes. |
c13182ef | 335 | Portable applications should avoid reliance on |
2adb3bd6 | 336 | bidirectional pipe semantics. |
1cb4e899 MK |
337 | .SS BUGS |
338 | Before Linux 4.9, some bugs affected the handling of the | |
339 | .IR pipe-user-pages-soft | |
340 | and | |
341 | .IR pipe-user-pages-hard | |
342 | limits when using the | |
343 | .BR fcntl (2) | |
344 | .BR F_SETPIPE_SZ | |
345 | operation to change a pipe's capacity: | |
346 | .\" These bugs where remedied by a series of patches, in particular, | |
347 | .\" commit b0b91d18e2e97b741b294af9333824ecc3fadfd8 and | |
348 | .\" commit a005ca0e6813e1d796a7422a7e31d8b8d6555df1 | |
349 | .IP (1) 5 | |
350 | When increasing the pipe capacity, the checks against the soft and | |
351 | hard limits were made against existing consumption, | |
352 | and excluded the memory required for the increased pipe capacity. | |
353 | The new increase in pipe capacity could then push the total | |
354 | memory used by the user for pipes (possibly far) over a limit. | |
355 | (This could also trigger the problem described next.) | |
356 | ||
357 | Starting with Linux 4.9, | |
358 | the limit checking includes the memory required for the new pipe capacity. | |
359 | .IP (2) | |
360 | The limit checks were performed even when the new pipe capacity was | |
361 | less than the existing pipe capacity. | |
362 | This could lead to problems if a user set a large pipe capacity, | |
363 | and then the limits were lowered, with the result that the user could | |
364 | no longer decrease the pipe capacity. | |
365 | ||
366 | Starting with Linux 4.9, checks against the limits | |
367 | are performed only when increasing a pipe's capacity; | |
368 | an unprivileged user can always decrease a pipe's capacity. | |
369 | .IP (3) | |
370 | The accounting and checking against the limits were done as follows: | |
371 | ||
372 | .RS | |
373 | .PD 0 | |
374 | .IP (a) 4 | |
375 | Test whether the user has exceeded the limit. | |
376 | .IP (b) | |
377 | Make the new pipe buffer allocation. | |
378 | .IP (c) | |
379 | Account new allocation against the limits. | |
380 | .PD | |
381 | .RE | |
382 | .IP | |
383 | This was racey. | |
384 | Multiple processes could pass point (a) simultaneously, | |
385 | and then allocate pipe buffers that were accounted for only in step (c), | |
386 | with the result that the user's pipe buffer | |
387 | allocation could be pushed over the limit. | |
388 | ||
389 | Starting with Linux 4.9, | |
390 | the accounting step is performed before doing the allocation, | |
391 | and the operation fails if the limit would be exceeded. | |
392 | .PP | |
393 | Before Linux 4.9, bugs similar to points (1) and (3) could also occur | |
394 | when the kernel allocated memory for a new pipe buffer; | |
395 | that is, when calling | |
396 | .BR pipe (2) | |
397 | and when opening a previously unopened FIFO. | |
47297adb | 398 | .SH SEE ALSO |
8a33c6e0 | 399 | .BR mkfifo (1), |
2adb3bd6 MK |
400 | .BR dup (2), |
401 | .BR fcntl (2), | |
402 | .BR open (2), | |
403 | .BR pipe (2), | |
404 | .BR poll (2), | |
405 | .BR select (2), | |
406 | .BR socketpair (2), | |
6b1b0c98 | 407 | .BR splice (2), |
2adb3bd6 MK |
408 | .BR stat (2), |
409 | .BR mkfifo (3), | |
af5b2ef2 MK |
410 | .BR epoll (7), |
411 | .BR fifo (7) |