1*58b9f456SAndroid Build Coastguard Worker<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" 2*58b9f456SAndroid Build Coastguard Worker "http://www.w3.org/TR/html4/strict.dtd"> 3*58b9f456SAndroid Build Coastguard Worker<!-- Material used from: HTML 4.01 specs: http://www.w3.org/TR/html401/ --> 4*58b9f456SAndroid Build Coastguard Worker<html> 5*58b9f456SAndroid Build Coastguard Worker<head> 6*58b9f456SAndroid Build Coastguard Worker <META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> 7*58b9f456SAndroid Build Coastguard Worker <title><atomic> design</title> 8*58b9f456SAndroid Build Coastguard Worker <link type="text/css" rel="stylesheet" href="menu.css"> 9*58b9f456SAndroid Build Coastguard Worker <link type="text/css" rel="stylesheet" href="content.css"> 10*58b9f456SAndroid Build Coastguard Worker</head> 11*58b9f456SAndroid Build Coastguard Worker 12*58b9f456SAndroid Build Coastguard Worker<body> 13*58b9f456SAndroid Build Coastguard Worker<div id="menu"> 14*58b9f456SAndroid Build Coastguard Worker <div> 15*58b9f456SAndroid Build Coastguard Worker <a href="https://llvm.org/">LLVM Home</a> 16*58b9f456SAndroid Build Coastguard Worker </div> 17*58b9f456SAndroid Build Coastguard Worker 18*58b9f456SAndroid Build Coastguard Worker <div class="submenu"> 19*58b9f456SAndroid Build Coastguard Worker <label>libc++ Info</label> 20*58b9f456SAndroid Build Coastguard Worker <a href="/index.html">About</a> 21*58b9f456SAndroid Build Coastguard Worker </div> 22*58b9f456SAndroid Build Coastguard Worker 23*58b9f456SAndroid Build Coastguard Worker <div class="submenu"> 24*58b9f456SAndroid Build Coastguard Worker <label>Quick Links</label> 25*58b9f456SAndroid Build Coastguard Worker <a href="https://lists.llvm.org/mailman/listinfo/cfe-dev">cfe-dev</a> 26*58b9f456SAndroid Build Coastguard Worker <a href="https://lists.llvm.org/mailman/listinfo/cfe-commits">cfe-commits</a> 27*58b9f456SAndroid Build Coastguard Worker <a href="https://bugs.llvm.org/">Bug Reports</a> 28*58b9f456SAndroid Build Coastguard Worker <a href="https://llvm.org/svn/llvm-project/libcxx/trunk/">Browse SVN</a> 29*58b9f456SAndroid Build Coastguard Worker <a href="https://llvm.org/viewvc/llvm-project/libcxx/trunk/">Browse ViewVC</a> 30*58b9f456SAndroid Build Coastguard Worker </div> 31*58b9f456SAndroid Build Coastguard Worker</div> 32*58b9f456SAndroid Build Coastguard Worker 33*58b9f456SAndroid Build Coastguard Worker<div id="content"> 34*58b9f456SAndroid Build Coastguard Worker <!--*********************************************************************--> 35*58b9f456SAndroid Build Coastguard Worker <h1><atomic> design</h1> 36*58b9f456SAndroid Build Coastguard Worker <!--*********************************************************************--> 37*58b9f456SAndroid Build Coastguard Worker 38*58b9f456SAndroid Build Coastguard Worker<p> 39*58b9f456SAndroid Build Coastguard WorkerThe <tt><atomic></tt> header is one of the most closely coupled headers to 40*58b9f456SAndroid Build Coastguard Workerthe compiler. Ideally when you invoke any function from 41*58b9f456SAndroid Build Coastguard Worker<tt><atomic></tt>, it should result in highly optimized assembly being 42*58b9f456SAndroid Build Coastguard Workerinserted directly into your application ... assembly that is not otherwise 43*58b9f456SAndroid Build Coastguard Workerrepresentable by higher level C or C++ expressions. The design of the libc++ 44*58b9f456SAndroid Build Coastguard Worker<tt><atomic></tt> header started with this goal in mind. A secondary, but 45*58b9f456SAndroid Build Coastguard Workerstill very important goal is that the compiler should have to do minimal work to 46*58b9f456SAndroid Build Coastguard Workerfacilitate the implementation of <tt><atomic></tt>. Without this second 47*58b9f456SAndroid Build Coastguard Workergoal, then practically speaking, the libc++ <tt><atomic></tt> header would 48*58b9f456SAndroid Build Coastguard Workerbe doomed to be a barely supported, second class citizen on almost every 49*58b9f456SAndroid Build Coastguard Workerplatform. 50*58b9f456SAndroid Build Coastguard Worker</p> 51*58b9f456SAndroid Build Coastguard Worker 52*58b9f456SAndroid Build Coastguard Worker<p>Goals:</p> 53*58b9f456SAndroid Build Coastguard Worker 54*58b9f456SAndroid Build Coastguard Worker<blockquote><ul> 55*58b9f456SAndroid Build Coastguard Worker<li>Optimal code generation for atomic operations</li> 56*58b9f456SAndroid Build Coastguard Worker<li>Minimal effort for the compiler to achieve goal 1 on any given platform</li> 57*58b9f456SAndroid Build Coastguard Worker<li>Conformance to the C++0X draft standard</li> 58*58b9f456SAndroid Build Coastguard Worker</ul></blockquote> 59*58b9f456SAndroid Build Coastguard Worker 60*58b9f456SAndroid Build Coastguard Worker<p> 61*58b9f456SAndroid Build Coastguard WorkerThe purpose of this document is to inform compiler writers what they need to do 62*58b9f456SAndroid Build Coastguard Workerto enable a high performance libc++ <tt><atomic></tt> with minimal effort. 63*58b9f456SAndroid Build Coastguard Worker</p> 64*58b9f456SAndroid Build Coastguard Worker 65*58b9f456SAndroid Build Coastguard Worker<h2>The minimal work that must be done for a conforming <tt><atomic></tt></h2> 66*58b9f456SAndroid Build Coastguard Worker 67*58b9f456SAndroid Build Coastguard Worker<p> 68*58b9f456SAndroid Build Coastguard WorkerThe only "atomic" operations that must actually be lock free in 69*58b9f456SAndroid Build Coastguard Worker<tt><atomic></tt> are represented by the following compiler intrinsics: 70*58b9f456SAndroid Build Coastguard Worker</p> 71*58b9f456SAndroid Build Coastguard Worker 72*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 73*58b9f456SAndroid Build Coastguard Worker__atomic_flag__ 74*58b9f456SAndroid Build Coastguard Worker__atomic_exchange_seq_cst(__atomic_flag__ volatile* obj, __atomic_flag__ desr) 75*58b9f456SAndroid Build Coastguard Worker{ 76*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 77*58b9f456SAndroid Build Coastguard Worker __atomic_flag__ result = *obj; 78*58b9f456SAndroid Build Coastguard Worker *obj = desr; 79*58b9f456SAndroid Build Coastguard Worker return result; 80*58b9f456SAndroid Build Coastguard Worker} 81*58b9f456SAndroid Build Coastguard Worker 82*58b9f456SAndroid Build Coastguard Workervoid 83*58b9f456SAndroid Build Coastguard Worker__atomic_store_seq_cst(__atomic_flag__ volatile* obj, __atomic_flag__ desr) 84*58b9f456SAndroid Build Coastguard Worker{ 85*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 86*58b9f456SAndroid Build Coastguard Worker *obj = desr; 87*58b9f456SAndroid Build Coastguard Worker} 88*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 89*58b9f456SAndroid Build Coastguard Worker 90*58b9f456SAndroid Build Coastguard Worker<p> 91*58b9f456SAndroid Build Coastguard WorkerWhere: 92*58b9f456SAndroid Build Coastguard Worker</p> 93*58b9f456SAndroid Build Coastguard Worker 94*58b9f456SAndroid Build Coastguard Worker<blockquote><ul> 95*58b9f456SAndroid Build Coastguard Worker<li> 96*58b9f456SAndroid Build Coastguard WorkerIf <tt>__has_feature(__atomic_flag)</tt> evaluates to 1 in the preprocessor then 97*58b9f456SAndroid Build Coastguard Workerthe compiler must define <tt>__atomic_flag__</tt> (e.g. as a typedef to 98*58b9f456SAndroid Build Coastguard Worker<tt>int</tt>). 99*58b9f456SAndroid Build Coastguard Worker</li> 100*58b9f456SAndroid Build Coastguard Worker<li> 101*58b9f456SAndroid Build Coastguard WorkerIf <tt>__has_feature(__atomic_flag)</tt> evaluates to 0 in the preprocessor then 102*58b9f456SAndroid Build Coastguard Workerthe library defines <tt>__atomic_flag__</tt> as a typedef to <tt>bool</tt>. 103*58b9f456SAndroid Build Coastguard Worker</li> 104*58b9f456SAndroid Build Coastguard Worker<li> 105*58b9f456SAndroid Build Coastguard Worker<p> 106*58b9f456SAndroid Build Coastguard WorkerTo communicate that the above intrinsics are available, the compiler must 107*58b9f456SAndroid Build Coastguard Workerarrange for <tt>__has_feature</tt> to return 1 when fed the intrinsic name 108*58b9f456SAndroid Build Coastguard Workerappended with an '_' and the mangled type name of <tt>__atomic_flag__</tt>. 109*58b9f456SAndroid Build Coastguard Worker</p> 110*58b9f456SAndroid Build Coastguard Worker<p> 111*58b9f456SAndroid Build Coastguard WorkerFor example if <tt>__atomic_flag__</tt> is <tt>unsigned int</tt>: 112*58b9f456SAndroid Build Coastguard Worker</p> 113*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 114*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_flag) == 1 115*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_j) == 1 116*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_store_seq_cst_j) == 1 117*58b9f456SAndroid Build Coastguard Worker 118*58b9f456SAndroid Build Coastguard Workertypedef unsigned int __atomic_flag__; 119*58b9f456SAndroid Build Coastguard Worker 120*58b9f456SAndroid Build Coastguard Workerunsigned int __atomic_exchange_seq_cst(unsigned int volatile*, unsigned int) 121*58b9f456SAndroid Build Coastguard Worker{ 122*58b9f456SAndroid Build Coastguard Worker // ... 123*58b9f456SAndroid Build Coastguard Worker} 124*58b9f456SAndroid Build Coastguard Worker 125*58b9f456SAndroid Build Coastguard Workervoid __atomic_store_seq_cst(unsigned int volatile*, unsigned int) 126*58b9f456SAndroid Build Coastguard Worker{ 127*58b9f456SAndroid Build Coastguard Worker // ... 128*58b9f456SAndroid Build Coastguard Worker} 129*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 130*58b9f456SAndroid Build Coastguard Worker</li> 131*58b9f456SAndroid Build Coastguard Worker</ul></blockquote> 132*58b9f456SAndroid Build Coastguard Worker 133*58b9f456SAndroid Build Coastguard Worker<p> 134*58b9f456SAndroid Build Coastguard WorkerThat's it! Compiler writers do the above and you've got a fully conforming 135*58b9f456SAndroid Build Coastguard Worker(though sub-par performance) <tt><atomic></tt> header! 136*58b9f456SAndroid Build Coastguard Worker</p> 137*58b9f456SAndroid Build Coastguard Worker 138*58b9f456SAndroid Build Coastguard Worker<h2>Recommended work for a higher performance <tt><atomic></tt></h2> 139*58b9f456SAndroid Build Coastguard Worker 140*58b9f456SAndroid Build Coastguard Worker<p> 141*58b9f456SAndroid Build Coastguard WorkerIt would be good if the above intrinsics worked with all integral types plus 142*58b9f456SAndroid Build Coastguard Worker<tt>void*</tt>. Because this may not be possible to do in a lock-free manner for 143*58b9f456SAndroid Build Coastguard Workerall integral types on all platforms, a compiler must communicate each type that 144*58b9f456SAndroid Build Coastguard Workeran intrinsic works with. For example if <tt>__atomic_exchange_seq_cst</tt> works 145*58b9f456SAndroid Build Coastguard Workerfor all types except for <tt>long long</tt> and <tt>unsigned long long</tt> 146*58b9f456SAndroid Build Coastguard Workerthen: 147*58b9f456SAndroid Build Coastguard Worker</p> 148*58b9f456SAndroid Build Coastguard Worker 149*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 150*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_b) == 1 // bool 151*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_c) == 1 // char 152*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_a) == 1 // signed char 153*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_h) == 1 // unsigned char 154*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_Ds) == 1 // char16_t 155*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_Di) == 1 // char32_t 156*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_w) == 1 // wchar_t 157*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_s) == 1 // short 158*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_t) == 1 // unsigned short 159*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_i) == 1 // int 160*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_j) == 1 // unsigned int 161*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_l) == 1 // long 162*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_m) == 1 // unsigned long 163*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_exchange_seq_cst_Pv) == 1 // void* 164*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 165*58b9f456SAndroid Build Coastguard Worker 166*58b9f456SAndroid Build Coastguard Worker<p> 167*58b9f456SAndroid Build Coastguard WorkerNote that only the <tt>__has_feature</tt> flag is decorated with the argument 168*58b9f456SAndroid Build Coastguard Workertype. The name of the compiler intrinsic is not decorated, but instead works 169*58b9f456SAndroid Build Coastguard Workerlike a C++ overloaded function. 170*58b9f456SAndroid Build Coastguard Worker</p> 171*58b9f456SAndroid Build Coastguard Worker 172*58b9f456SAndroid Build Coastguard Worker<p> 173*58b9f456SAndroid Build Coastguard WorkerAdditionally there are other intrinsics besides 174*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_exchange_seq_cst</tt> and <tt>__atomic_store_seq_cst</tt>. They 175*58b9f456SAndroid Build Coastguard Workerare optional. But if the compiler can generate faster code than provided by the 176*58b9f456SAndroid Build Coastguard Workerlibrary, then clients will benefit from the compiler writer's expertise and 177*58b9f456SAndroid Build Coastguard Workerknowledge of the targeted platform. 178*58b9f456SAndroid Build Coastguard Worker</p> 179*58b9f456SAndroid Build Coastguard Worker 180*58b9f456SAndroid Build Coastguard Worker<p> 181*58b9f456SAndroid Build Coastguard WorkerBelow is the complete list of <i>sequentially consistent</i> intrinsics, and 182*58b9f456SAndroid Build Coastguard Workertheir library implementations. Template syntax is used to indicate the desired 183*58b9f456SAndroid Build Coastguard Workeroverloading for integral and void* types. The template does not represent a 184*58b9f456SAndroid Build Coastguard Workerrequirement that the intrinsic operate on <em>any</em> type! 185*58b9f456SAndroid Build Coastguard Worker</p> 186*58b9f456SAndroid Build Coastguard Worker 187*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 188*58b9f456SAndroid Build Coastguard WorkerT is one of: bool, char, signed char, unsigned char, short, unsigned short, 189*58b9f456SAndroid Build Coastguard Worker int, unsigned int, long, unsigned long, 190*58b9f456SAndroid Build Coastguard Worker long long, unsigned long long, char16_t, char32_t, wchar_t, void* 191*58b9f456SAndroid Build Coastguard Worker 192*58b9f456SAndroid Build Coastguard Workertemplate <class T> 193*58b9f456SAndroid Build Coastguard WorkerT 194*58b9f456SAndroid Build Coastguard Worker__atomic_load_seq_cst(T const volatile* obj) 195*58b9f456SAndroid Build Coastguard Worker{ 196*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 197*58b9f456SAndroid Build Coastguard Worker return *obj; 198*58b9f456SAndroid Build Coastguard Worker} 199*58b9f456SAndroid Build Coastguard Worker 200*58b9f456SAndroid Build Coastguard Workertemplate <class T> 201*58b9f456SAndroid Build Coastguard Workervoid 202*58b9f456SAndroid Build Coastguard Worker__atomic_store_seq_cst(T volatile* obj, T desr) 203*58b9f456SAndroid Build Coastguard Worker{ 204*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 205*58b9f456SAndroid Build Coastguard Worker *obj = desr; 206*58b9f456SAndroid Build Coastguard Worker} 207*58b9f456SAndroid Build Coastguard Worker 208*58b9f456SAndroid Build Coastguard Workertemplate <class T> 209*58b9f456SAndroid Build Coastguard WorkerT 210*58b9f456SAndroid Build Coastguard Worker__atomic_exchange_seq_cst(T volatile* obj, T desr) 211*58b9f456SAndroid Build Coastguard Worker{ 212*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 213*58b9f456SAndroid Build Coastguard Worker T r = *obj; 214*58b9f456SAndroid Build Coastguard Worker *obj = desr; 215*58b9f456SAndroid Build Coastguard Worker return r; 216*58b9f456SAndroid Build Coastguard Worker} 217*58b9f456SAndroid Build Coastguard Worker 218*58b9f456SAndroid Build Coastguard Workertemplate <class T> 219*58b9f456SAndroid Build Coastguard Workerbool 220*58b9f456SAndroid Build Coastguard Worker__atomic_compare_exchange_strong_seq_cst_seq_cst(T volatile* obj, T* exp, T desr) 221*58b9f456SAndroid Build Coastguard Worker{ 222*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 223*58b9f456SAndroid Build Coastguard Worker if (std::memcmp(const_cast<T*>(obj), exp, sizeof(T)) == 0) 224*58b9f456SAndroid Build Coastguard Worker { 225*58b9f456SAndroid Build Coastguard Worker std::memcpy(const_cast<T*>(obj), &desr, sizeof(T)); 226*58b9f456SAndroid Build Coastguard Worker return true; 227*58b9f456SAndroid Build Coastguard Worker } 228*58b9f456SAndroid Build Coastguard Worker std::memcpy(exp, const_cast<T*>(obj), sizeof(T)); 229*58b9f456SAndroid Build Coastguard Worker return false; 230*58b9f456SAndroid Build Coastguard Worker} 231*58b9f456SAndroid Build Coastguard Worker 232*58b9f456SAndroid Build Coastguard Workertemplate <class T> 233*58b9f456SAndroid Build Coastguard Workerbool 234*58b9f456SAndroid Build Coastguard Worker__atomic_compare_exchange_weak_seq_cst_seq_cst(T volatile* obj, T* exp, T desr) 235*58b9f456SAndroid Build Coastguard Worker{ 236*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 237*58b9f456SAndroid Build Coastguard Worker if (std::memcmp(const_cast<T*>(obj), exp, sizeof(T)) == 0) 238*58b9f456SAndroid Build Coastguard Worker { 239*58b9f456SAndroid Build Coastguard Worker std::memcpy(const_cast<T*>(obj), &desr, sizeof(T)); 240*58b9f456SAndroid Build Coastguard Worker return true; 241*58b9f456SAndroid Build Coastguard Worker } 242*58b9f456SAndroid Build Coastguard Worker std::memcpy(exp, const_cast<T*>(obj), sizeof(T)); 243*58b9f456SAndroid Build Coastguard Worker return false; 244*58b9f456SAndroid Build Coastguard Worker} 245*58b9f456SAndroid Build Coastguard Worker 246*58b9f456SAndroid Build Coastguard WorkerT is one of: char, signed char, unsigned char, short, unsigned short, 247*58b9f456SAndroid Build Coastguard Worker int, unsigned int, long, unsigned long, 248*58b9f456SAndroid Build Coastguard Worker long long, unsigned long long, char16_t, char32_t, wchar_t 249*58b9f456SAndroid Build Coastguard Worker 250*58b9f456SAndroid Build Coastguard Workertemplate <class T> 251*58b9f456SAndroid Build Coastguard WorkerT 252*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_add_seq_cst(T volatile* obj, T operand) 253*58b9f456SAndroid Build Coastguard Worker{ 254*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 255*58b9f456SAndroid Build Coastguard Worker T r = *obj; 256*58b9f456SAndroid Build Coastguard Worker *obj += operand; 257*58b9f456SAndroid Build Coastguard Worker return r; 258*58b9f456SAndroid Build Coastguard Worker} 259*58b9f456SAndroid Build Coastguard Worker 260*58b9f456SAndroid Build Coastguard Workertemplate <class T> 261*58b9f456SAndroid Build Coastguard WorkerT 262*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_sub_seq_cst(T volatile* obj, T operand) 263*58b9f456SAndroid Build Coastguard Worker{ 264*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 265*58b9f456SAndroid Build Coastguard Worker T r = *obj; 266*58b9f456SAndroid Build Coastguard Worker *obj -= operand; 267*58b9f456SAndroid Build Coastguard Worker return r; 268*58b9f456SAndroid Build Coastguard Worker} 269*58b9f456SAndroid Build Coastguard Worker 270*58b9f456SAndroid Build Coastguard Workertemplate <class T> 271*58b9f456SAndroid Build Coastguard WorkerT 272*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_and_seq_cst(T volatile* obj, T operand) 273*58b9f456SAndroid Build Coastguard Worker{ 274*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 275*58b9f456SAndroid Build Coastguard Worker T r = *obj; 276*58b9f456SAndroid Build Coastguard Worker *obj &= operand; 277*58b9f456SAndroid Build Coastguard Worker return r; 278*58b9f456SAndroid Build Coastguard Worker} 279*58b9f456SAndroid Build Coastguard Worker 280*58b9f456SAndroid Build Coastguard Workertemplate <class T> 281*58b9f456SAndroid Build Coastguard WorkerT 282*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_or_seq_cst(T volatile* obj, T operand) 283*58b9f456SAndroid Build Coastguard Worker{ 284*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 285*58b9f456SAndroid Build Coastguard Worker T r = *obj; 286*58b9f456SAndroid Build Coastguard Worker *obj |= operand; 287*58b9f456SAndroid Build Coastguard Worker return r; 288*58b9f456SAndroid Build Coastguard Worker} 289*58b9f456SAndroid Build Coastguard Worker 290*58b9f456SAndroid Build Coastguard Workertemplate <class T> 291*58b9f456SAndroid Build Coastguard WorkerT 292*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_xor_seq_cst(T volatile* obj, T operand) 293*58b9f456SAndroid Build Coastguard Worker{ 294*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 295*58b9f456SAndroid Build Coastguard Worker T r = *obj; 296*58b9f456SAndroid Build Coastguard Worker *obj ^= operand; 297*58b9f456SAndroid Build Coastguard Worker return r; 298*58b9f456SAndroid Build Coastguard Worker} 299*58b9f456SAndroid Build Coastguard Worker 300*58b9f456SAndroid Build Coastguard Workervoid* 301*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_add_seq_cst(void* volatile* obj, ptrdiff_t operand) 302*58b9f456SAndroid Build Coastguard Worker{ 303*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 304*58b9f456SAndroid Build Coastguard Worker void* r = *obj; 305*58b9f456SAndroid Build Coastguard Worker (char*&)(*obj) += operand; 306*58b9f456SAndroid Build Coastguard Worker return r; 307*58b9f456SAndroid Build Coastguard Worker} 308*58b9f456SAndroid Build Coastguard Worker 309*58b9f456SAndroid Build Coastguard Workervoid* 310*58b9f456SAndroid Build Coastguard Worker__atomic_fetch_sub_seq_cst(void* volatile* obj, ptrdiff_t operand) 311*58b9f456SAndroid Build Coastguard Worker{ 312*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 313*58b9f456SAndroid Build Coastguard Worker void* r = *obj; 314*58b9f456SAndroid Build Coastguard Worker (char*&)(*obj) -= operand; 315*58b9f456SAndroid Build Coastguard Worker return r; 316*58b9f456SAndroid Build Coastguard Worker} 317*58b9f456SAndroid Build Coastguard Worker 318*58b9f456SAndroid Build Coastguard Workervoid __atomic_thread_fence_seq_cst() 319*58b9f456SAndroid Build Coastguard Worker{ 320*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 321*58b9f456SAndroid Build Coastguard Worker} 322*58b9f456SAndroid Build Coastguard Worker 323*58b9f456SAndroid Build Coastguard Workervoid __atomic_signal_fence_seq_cst() 324*58b9f456SAndroid Build Coastguard Worker{ 325*58b9f456SAndroid Build Coastguard Worker unique_lock<mutex> _(some_mutex); 326*58b9f456SAndroid Build Coastguard Worker} 327*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 328*58b9f456SAndroid Build Coastguard Worker 329*58b9f456SAndroid Build Coastguard Worker<p> 330*58b9f456SAndroid Build Coastguard WorkerOne should consult the (currently draft) 331*58b9f456SAndroid Build Coastguard Worker<a href="https://wg21.link/n3126">C++ standard</a> 332*58b9f456SAndroid Build Coastguard Workerfor the details of the definitions for these operations. For example 333*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_weak_seq_cst_seq_cst</tt> is allowed to fail 334*58b9f456SAndroid Build Coastguard Workerspuriously while <tt>__atomic_compare_exchange_strong_seq_cst_seq_cst</tt> is 335*58b9f456SAndroid Build Coastguard Workernot. 336*58b9f456SAndroid Build Coastguard Worker</p> 337*58b9f456SAndroid Build Coastguard Worker 338*58b9f456SAndroid Build Coastguard Worker<p> 339*58b9f456SAndroid Build Coastguard WorkerIf on your platform the lock-free definition of 340*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_weak_seq_cst_seq_cst</tt> would be the same as 341*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_strong_seq_cst_seq_cst</tt>, you may omit the 342*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_weak_seq_cst_seq_cst</tt> intrinsic without a 343*58b9f456SAndroid Build Coastguard Workerperformance cost. The library will prefer your implementation of 344*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_strong_seq_cst_seq_cst</tt> over its own 345*58b9f456SAndroid Build Coastguard Workerdefinition for implementing 346*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_weak_seq_cst_seq_cst</tt>. That is, the library 347*58b9f456SAndroid Build Coastguard Workerwill arrange for <tt>__atomic_compare_exchange_weak_seq_cst_seq_cst</tt> to call 348*58b9f456SAndroid Build Coastguard Worker<tt>__atomic_compare_exchange_strong_seq_cst_seq_cst</tt> if you supply an 349*58b9f456SAndroid Build Coastguard Workerintrinsic for the strong version but not the weak. 350*58b9f456SAndroid Build Coastguard Worker</p> 351*58b9f456SAndroid Build Coastguard Worker 352*58b9f456SAndroid Build Coastguard Worker<h2>Taking advantage of weaker memory synchronization</h2> 353*58b9f456SAndroid Build Coastguard Worker 354*58b9f456SAndroid Build Coastguard Worker<p> 355*58b9f456SAndroid Build Coastguard WorkerSo far all of the intrinsics presented require a <em>sequentially 356*58b9f456SAndroid Build Coastguard Workerconsistent</em> memory ordering. That is, no loads or stores can move across 357*58b9f456SAndroid Build Coastguard Workerthe operation (just as if the library had locked that internal mutex). But 358*58b9f456SAndroid Build Coastguard Worker<tt><atomic></tt> supports weaker memory ordering operations. In all, 359*58b9f456SAndroid Build Coastguard Workerthere are six memory orderings (listed here from strongest to weakest): 360*58b9f456SAndroid Build Coastguard Worker</p> 361*58b9f456SAndroid Build Coastguard Worker 362*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 363*58b9f456SAndroid Build Coastguard Workermemory_order_seq_cst 364*58b9f456SAndroid Build Coastguard Workermemory_order_acq_rel 365*58b9f456SAndroid Build Coastguard Workermemory_order_release 366*58b9f456SAndroid Build Coastguard Workermemory_order_acquire 367*58b9f456SAndroid Build Coastguard Workermemory_order_consume 368*58b9f456SAndroid Build Coastguard Workermemory_order_relaxed 369*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 370*58b9f456SAndroid Build Coastguard Worker 371*58b9f456SAndroid Build Coastguard Worker<p> 372*58b9f456SAndroid Build Coastguard Worker(See the 373*58b9f456SAndroid Build Coastguard Worker<a href="https://wg21.link/n3126">C++ standard</a> 374*58b9f456SAndroid Build Coastguard Workerfor the detailed definitions of each of these orderings). 375*58b9f456SAndroid Build Coastguard Worker</p> 376*58b9f456SAndroid Build Coastguard Worker 377*58b9f456SAndroid Build Coastguard Worker<p> 378*58b9f456SAndroid Build Coastguard WorkerOn some platforms, the compiler vendor can offer some or even all of the above 379*58b9f456SAndroid Build Coastguard Workerintrinsics at one or more weaker levels of memory synchronization. This might 380*58b9f456SAndroid Build Coastguard Workerlead for example to not issuing an <tt>mfence</tt> instruction on the x86. 381*58b9f456SAndroid Build Coastguard Worker</p> 382*58b9f456SAndroid Build Coastguard Worker 383*58b9f456SAndroid Build Coastguard Worker<p> 384*58b9f456SAndroid Build Coastguard WorkerIf the compiler does not offer any given operation, at any given memory ordering 385*58b9f456SAndroid Build Coastguard Workerlevel, the library will automatically attempt to call the next highest memory 386*58b9f456SAndroid Build Coastguard Workerordering operation. This continues up to <tt>seq_cst</tt>, and if that doesn't 387*58b9f456SAndroid Build Coastguard Workerexist, then the library takes over and does the job with a <tt>mutex</tt>. This 388*58b9f456SAndroid Build Coastguard Workeris a compile-time search & selection operation. At run time, the 389*58b9f456SAndroid Build Coastguard Workerapplication will only see the few inlined assembly instructions for the selected 390*58b9f456SAndroid Build Coastguard Workerintrinsic. 391*58b9f456SAndroid Build Coastguard Worker</p> 392*58b9f456SAndroid Build Coastguard Worker 393*58b9f456SAndroid Build Coastguard Worker<p> 394*58b9f456SAndroid Build Coastguard WorkerEach intrinsic is appended with the 7-letter name of the memory ordering it 395*58b9f456SAndroid Build Coastguard Workeraddresses. For example a <tt>load</tt> with <tt>relaxed</tt> ordering is 396*58b9f456SAndroid Build Coastguard Workerdefined by: 397*58b9f456SAndroid Build Coastguard Worker</p> 398*58b9f456SAndroid Build Coastguard Worker 399*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 400*58b9f456SAndroid Build Coastguard WorkerT __atomic_load_relaxed(const volatile T* obj); 401*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 402*58b9f456SAndroid Build Coastguard Worker 403*58b9f456SAndroid Build Coastguard Worker<p> 404*58b9f456SAndroid Build Coastguard WorkerAnd announced with: 405*58b9f456SAndroid Build Coastguard Worker</p> 406*58b9f456SAndroid Build Coastguard Worker 407*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 408*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_load_relaxed_b) == 1 // bool 409*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_load_relaxed_c) == 1 // char 410*58b9f456SAndroid Build Coastguard Worker__has_feature(__atomic_load_relaxed_a) == 1 // signed char 411*58b9f456SAndroid Build Coastguard Worker... 412*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 413*58b9f456SAndroid Build Coastguard Worker 414*58b9f456SAndroid Build Coastguard Worker<p> 415*58b9f456SAndroid Build Coastguard WorkerThe <tt>__atomic_compare_exchange_strong(weak)</tt> intrinsics are parameterized 416*58b9f456SAndroid Build Coastguard Workeron two memory orderings. The first ordering applies when the operation returns 417*58b9f456SAndroid Build Coastguard Worker<tt>true</tt> and the second ordering applies when the operation returns 418*58b9f456SAndroid Build Coastguard Worker<tt>false</tt>. 419*58b9f456SAndroid Build Coastguard Worker</p> 420*58b9f456SAndroid Build Coastguard Worker 421*58b9f456SAndroid Build Coastguard Worker<p> 422*58b9f456SAndroid Build Coastguard WorkerNot every memory ordering is appropriate for every operation. <tt>exchange</tt> 423*58b9f456SAndroid Build Coastguard Workerand the <tt>fetch_<i>op</i></tt> operations support all 6. But <tt>load</tt> 424*58b9f456SAndroid Build Coastguard Workeronly supports <tt>relaxed</tt>, <tt>consume</tt>, <tt>acquire</tt> and <tt>seq_cst</tt>. 425*58b9f456SAndroid Build Coastguard Worker<tt>store</tt> 426*58b9f456SAndroid Build Coastguard Workeronly supports <tt>relaxed</tt>, <tt>release</tt>, and <tt>seq_cst</tt>. The 427*58b9f456SAndroid Build Coastguard Worker<tt>compare_exchange</tt> operations support the following 16 combinations out 428*58b9f456SAndroid Build Coastguard Workerof the possible 36: 429*58b9f456SAndroid Build Coastguard Worker</p> 430*58b9f456SAndroid Build Coastguard Worker 431*58b9f456SAndroid Build Coastguard Worker<blockquote><pre> 432*58b9f456SAndroid Build Coastguard Workerrelaxed_relaxed 433*58b9f456SAndroid Build Coastguard Workerconsume_relaxed 434*58b9f456SAndroid Build Coastguard Workerconsume_consume 435*58b9f456SAndroid Build Coastguard Workeracquire_relaxed 436*58b9f456SAndroid Build Coastguard Workeracquire_consume 437*58b9f456SAndroid Build Coastguard Workeracquire_acquire 438*58b9f456SAndroid Build Coastguard Workerrelease_relaxed 439*58b9f456SAndroid Build Coastguard Workerrelease_consume 440*58b9f456SAndroid Build Coastguard Workerrelease_acquire 441*58b9f456SAndroid Build Coastguard Workeracq_rel_relaxed 442*58b9f456SAndroid Build Coastguard Workeracq_rel_consume 443*58b9f456SAndroid Build Coastguard Workeracq_rel_acquire 444*58b9f456SAndroid Build Coastguard Workerseq_cst_relaxed 445*58b9f456SAndroid Build Coastguard Workerseq_cst_consume 446*58b9f456SAndroid Build Coastguard Workerseq_cst_acquire 447*58b9f456SAndroid Build Coastguard Workerseq_cst_seq_cst 448*58b9f456SAndroid Build Coastguard Worker</pre></blockquote> 449*58b9f456SAndroid Build Coastguard Worker 450*58b9f456SAndroid Build Coastguard Worker<p> 451*58b9f456SAndroid Build Coastguard WorkerAgain, the compiler supplies intrinsics only for the strongest orderings where 452*58b9f456SAndroid Build Coastguard Workerit can make a difference. The library takes care of calling the weakest 453*58b9f456SAndroid Build Coastguard Workersupplied intrinsic that is as strong or stronger than the customer asked for. 454*58b9f456SAndroid Build Coastguard Worker</p> 455*58b9f456SAndroid Build Coastguard Worker 456*58b9f456SAndroid Build Coastguard Worker</div> 457*58b9f456SAndroid Build Coastguard Worker</body> 458*58b9f456SAndroid Build Coastguard Worker</html> 459