Lines Matching +full:line +full:- +full:orders
1 This document gives an overview of the categories of memory-ordering
2 operations provided by the Linux-kernel memory model (LKMM).
8 This section lists LKMM's three top-level categories of memory-ordering
11 1. Barriers (also known as "fences"). A barrier orders some or
38 b. Read-modify-write (RMW) ordering augmentation barriers.
56 --------------------
58 The Linux-kernel primitives that provide full ordering include:
62 o Value-returning RMW atomic operations whose names do not end in
65 o RCU's grace-period primitives.
67 First, the smp_mb() full memory barrier orders all of the CPU's prior
79 memory-ordering primitives. It is surprisingly hard to remember their
83 operations include value-returning RMW atomic operations (that is, those
84 with non-void return types) whose names do not end in _acquire, _release,
98 before any operation in a higher-numbered partition.
100 In contrast, non-value-returning RMW atomic operations (that is, those
102 value-returning RMW atomic operations whose names end in _relaxed.
105 atomic_xchg_relaxed(). Similarly, value-returning non-RMW atomic
109 Value-returning RMW atomic operations whose names end in _acquire or
113 Finally, RCU's grace-period primitives provide full ordering. These
115 synchronize_srcu() and so on. However, these primitives have orders
117 Furthermore, RCU's grace-period primitives can only be invoked in
118 sleepable contexts. Therefore, RCU's grace-period primitives are
119 typically instead used to provide ordering against RCU read-side critical
122 to also rely on its additional full-memory-barrier semantics. Just please
127 ----------------------------------
129 As noted in the previous section, non-value-returning RMW operations
174 --------------------
187 Note that smp_wmb() might fail to provide ordering for unmarked C-language
188 stores because profile-driven optimization could determine that the
199 Therefore, if you need to use smp_wmb() with unmarked C-language writes,
206 -------------------
220 ----------------
223 prohibits compiler code-motion optimizations that might move memory
238 Note that the barriers discussed previously use barrier() or its low-level
251 c. RCU read-side ordering.
259 ------------------
262 rcu_assign_pointer(), and value-returning RMW operations whose names
266 For example, use of smp_store_release() saves a line compared to the
303 for an RCU-protected pointer in "sparse" runs.
305 o Value-returning RMW operations whose names end in _release,
308 memory-store portion of the RMW operation, and not against the
309 memory-load portion. Note also that conditional operations such
318 ------------------
321 and value-returning RMW operations whose names end in _acquire. These
325 smp_load_acquire() saves a line compared to the smp_rmb() example above:
342 o Value-returning RMW operations whose names end in _acquire,
345 memory-load portion of the RMW operation, and not against the
346 memory-store portion. Note also that conditional operations
371 RCU Read-Side Ordering
372 ----------------------
374 This category includes read-side markers such as rcu_read_lock()
375 and rcu_read_unlock() as well as pointer-traversal primitives such as
379 for RCU read-side critical sections incur very low overhead because
380 they interact only with the corresponding grace-period primitives.
389 RCU's pointer-traversal primitives, including rcu_dereference() and
396 A call to rcu_dereference() for a given RCU-protected pointer is
413 --------------------
419 control-flow instructions such as comparisons and conditional branches.
435 "control-dependencies.txt" file for more information.
445 b. Unmarked C-language accesses.
449 ---------------------------
475 o Unordered RMW atomic operations. These are non-value-returning
477 _release, and also value-returning RMW operations whose names
495 Unmarked C-Language Accesses
496 ----------------------------
498 Unmarked C-language accesses are normal variable accesses to normal
502 might (and sometimes does) split a plain C-language store into multiple
507 Unmarked C-language accesses are unordered, and are also subject to
509 concurrent code. It is possible to used unmarked C-language accesses for
511 is required on an ongoing basis. The compiler-constraining barrier()
514 C-language accesses requires careful attention to not just your code,
520 But there are some ways of using unmarked C-language accesses for shared
527 unmarked C-language access and (2) when at least one of those
531 as reader-writer locks or sequence locks.
553 Used properly, unmarked C-language accesses can reduce overhead on