1 .\" Copyright 2015-2017 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
3 .\" SPDX-License-Identifier: Linux-man-pages-copyleft
5 .TH MEMBARRIER 2 2021-08-27 "Linux man-pages (unreleased)"
7 membarrier \- issue memory barriers on a set of threads
10 .RI ( libc ", " \-lc )
14 .BR "#include <linux/membarrier.h>" \
15 " /* Definition of " MEMBARRIER_* " constants */"
16 .BR "#include <sys/syscall.h>" " /* Definition of " SYS_* " constants */"
17 .B #include <unistd.h>
19 .BI "int syscall(SYS_membarrier, int " cmd ", unsigned int " flags \
24 glibc provides no wrapper for
26 necessitating the use of
31 system call helps reducing the overhead of the memory barrier
32 instructions required to order memory accesses on multi-core systems.
33 However, this system call is heavier than a memory barrier, so using it
36 as simple as replacing memory barriers with this
37 system call, but requires understanding of the details below.
39 Use of memory barriers needs to be done taking into account that a
40 memory barrier always needs to be either matched with its memory barrier
41 counterparts, or that the architecture's memory model doesn't require the
44 There are cases where one side of the matching barriers (which we will
45 refer to as "fast side") is executed much more often than the other
46 (which we will refer to as "slow side").
47 This is a prime target for the use of
49 The key idea is to replace, for these matching
50 barriers, the fast-side memory barriers by simple compiler barriers,
55 asm volatile ("" : : : "memory")
59 and replace the slow-side memory barriers by calls to
62 This will add overhead to the slow side, and remove overhead from the
63 fast side, thus resulting in an overall performance increase as long as
64 the slow side is infrequent enough that the overhead of the
66 calls does not outweigh the performance gain on the fast side.
70 argument is one of the following:
72 .BR MEMBARRIER_CMD_QUERY " (since Linux 4.3)"
73 Query the set of supported commands.
74 The return value of the call is a bit mask of supported
76 .BR MEMBARRIER_CMD_QUERY ,
77 which has the value 0,
78 is not itself included in this bit mask.
79 This command is always supported (on kernels where
83 .BR MEMBARRIER_CMD_GLOBAL " (since Linux 4.16)"
84 Ensure that all threads from all processes on the system pass through a
85 state where all memory accesses to user-space addresses match program
86 order between entry to and return from the
89 All threads on the system are targeted by this command.
91 .BR MEMBARRIER_CMD_GLOBAL_EXPEDITED " (since Linux 4.16)"
92 Execute a memory barrier on all running threads of all processes that
93 previously registered with
94 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED .
96 Upon return from the system call, the calling thread has a guarantee that all
97 running threads have passed through a state where all memory accesses to
98 user-space addresses match program order between entry to and return
99 from the system call (non-running threads are de facto in such a state).
100 This guarantee is provided only for the threads of processes that
101 previously registered with
102 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED .
104 Given that registration is about the intent to receive the barriers, it
106 .B MEMBARRIER_CMD_GLOBAL_EXPEDITED
107 from a process that has not employed
108 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED .
110 The "expedited" commands complete faster than the non-expedited ones;
111 they never block, but have the downside of causing extra overhead.
113 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED " (since Linux 4.16)"
114 Register the process's intent to receive
115 .B MEMBARRIER_CMD_GLOBAL_EXPEDITED
118 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED " (since Linux 4.14)"
119 Execute a memory barrier on each running thread belonging to the same
120 process as the calling thread.
122 Upon return from the system call, the calling
123 thread has a guarantee that all its running thread siblings have passed
124 through a state where all memory accesses to user-space addresses match
125 program order between entry to and return from the system call
126 (non-running threads are de facto in such a state).
127 This guarantee is provided only for threads in
128 the same process as the calling thread.
130 The "expedited" commands complete faster than the non-expedited ones;
131 they never block, but have the downside of causing extra overhead.
133 A process must register its intent to use the private
134 expedited command prior to using it.
136 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED " (since Linux 4.14)"
137 Register the process's intent to use
138 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED .
140 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE " (since Linux 4.16)"
141 In addition to providing the memory ordering guarantees described in
142 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED ,
143 upon return from system call the calling thread has a guarantee that all its
144 running thread siblings have executed a core serializing instruction.
145 This guarantee is provided only for threads in
146 the same process as the calling thread.
148 The "expedited" commands complete faster than the non-expedited ones,
149 they never block, but have the downside of causing extra overhead.
151 A process must register its intent to use the private expedited sync
152 core command prior to using it.
154 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE " (since Linux 4.16)"
155 Register the process's intent to use
156 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE .
158 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ " (since Linux 5.10)"
159 Ensure the caller thread, upon return from system call, that all its
160 running thread siblings have any currently running rseq critical sections
166 .BR MEMBARRIER_CMD_FLAG_CPU ,
167 then this operation is performed only on CPU indicated by
169 This guarantee is provided only for threads in
170 the same process as the calling thread.
172 RSEQ membarrier is only available in the "private expedited" form.
174 A process must register its intent to use the private expedited rseq
175 command prior to using it.
177 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ " (since Linux 5.10)"
178 Register the process's intent to use
179 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ .
181 .BR MEMBARRIER_CMD_SHARED " (since Linux 4.3)"
183 .B MEMBARRIER_CMD_GLOBAL
184 that exists for header backward compatibility.
188 argument must be specified as 0 unless the command is
189 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ ,
193 .BR MEMBARRIER_CMD_FLAG_CPU .
197 argument is ignored unless
200 .BR MEMBARRIER_CMD_FLAG_CPU ,
201 in which case it must specify the CPU targeted by this membarrier
204 All memory accesses performed in program order from each targeted thread
205 are guaranteed to be ordered with respect to
208 If we use the semantic
210 to represent a compiler barrier forcing memory
211 accesses to be performed in program order across the barrier, and
213 to represent explicit memory barriers forcing full memory
214 ordering across the barrier, we have the following ordering table for
220 The pair ordering is detailed as (O: ordered, X: not ordered):
225 \& barrier() smp_mb() membarrier()
233 .B MEMBARRIER_CMD_QUERY
234 operation returns a bit mask of supported commands, and the
235 .BR MEMBARRIER_CMD_GLOBAL ,
236 .BR MEMBARRIER_CMD_GLOBAL_EXPEDITED ,
237 .BR MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED ,
238 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED ,
239 .BR MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED ,
240 .BR MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE ,
242 .B MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE
243 operations return zero.
244 On error, \-1 is returned,
247 is set to indicate the error.
249 For a given command, with
251 set to 0, this system call is
252 guaranteed to always return the same value until reboot.
253 Further calls with the same arguments will lead to the same result.
256 set to 0, error handling is required only for the first call to
265 .B MEMBARRIER_CMD_GLOBAL
266 command is disabled because the
268 CPU parameter has been set, or the
269 .B MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE
271 .B MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE
272 commands are not implemented by the architecture.
277 system call is not implemented by this kernel.
280 The current process was not registered prior to using private expedited
285 system call was added in Linux 4.3.
287 Before Linux 5.10, the prototype for
293 .BI "int membarrier(int " cmd ", int " flags );
300 .\" FIXME See if the following syscalls make it into Linux 4.15 or later
304 A memory barrier instruction is part of the instruction set of
305 architectures with weakly ordered memory models.
307 accesses prior to the barrier and after the barrier with respect to
308 matching barriers on other cores.
309 For instance, a load fence can order
310 loads prior to and following that fence with respect to stores ordered
313 Program order is the order in which instructions are ordered in the
314 program assembly code.
318 can be useful include implementations
319 of Read-Copy-Update libraries and garbage collectors.
321 Assuming a multithreaded application where "fast_path()" is executed
322 very frequently, and where "slow_path()" is executed infrequently, the
323 following code (x86) can be transformed using
327 .\" SRC BEGIN (membarrier.c)
331 static volatile int a, b;
334 fast_path(int *read_b)
337 asm volatile ("mfence" : : : "memory");
342 slow_path(int *read_a)
345 asm volatile ("mfence" : : : "memory");
355 * Real applications would call fast_path() and slow_path()
356 * from different threads. Call those from main() to keep
357 * this example short.
364 * read_b == 0 implies read_a == 1 and
365 * read_a == 0 implies read_b == 1.
368 if (read_b == 0 && read_a == 0)
377 The code above transformed to use
387 #include <sys/syscall.h>
388 #include <linux/membarrier.h>
390 static volatile int a, b;
393 membarrier(int cmd, unsigned int flags, int cpu_id)
395 return syscall(__NR_membarrier, cmd, flags, cpu_id);
399 init_membarrier(void)
403 /* Check that membarrier() is supported. */
405 ret = membarrier(MEMBARRIER_CMD_QUERY, 0, 0);
407 perror("membarrier");
411 if (!(ret & MEMBARRIER_CMD_GLOBAL)) {
413 "membarrier does not support MEMBARRIER_CMD_GLOBAL\en");
421 fast_path(int *read_b)
424 asm volatile ("" : : : "memory");
429 slow_path(int *read_a)
432 membarrier(MEMBARRIER_CMD_GLOBAL, 0, 0);
437 main(int argc, char *argv[])
441 if (init_membarrier())
445 * Real applications would call fast_path() and slow_path()
446 * from different threads. Call those from main() to keep
447 * this example short.
454 * read_b == 0 implies read_a == 1 and
455 * read_a == 0 implies read_b == 1.
458 if (read_b == 0 && read_a == 0)