7  * Writer lock-stealing by Alex Shi <[email protected]>
41  * When the rwsem is reader-owned and a spinning writer has timed out,
44  * When a writer acquires a rwsem, it puts its task_struct pointer
59  * is previously owned by a writer and the following conditions are met:
60  *  - rwsem is not currently writer owned
84  * Bit  0    - writer locked bit
93  * Bit  0    - writer locked bit
106  * atomic_long_cmpxchg() will be used to obtain writer lock.
113  * For all the above cases, wait_lock will be held. A writer must also
292  * by a writer.
301  * and a writer has just released the lock. So another writer may steal
428 			 * Mark writer at the front of the queue for wakeup.  in rwsem_mark_wake()
432 			 * will notice the queued writer.  in rwsem_mark_wake()
450 	 * so we can bail out early if a writer stole the lock.  in rwsem_mark_wake()
571  * writer slowpaths with wait_lock held. It releases the wait_lock and
670  *   OWNER_WRITER: when owner changes and is a writer
686  * Try to acquire write lock before the writer has been put on wait queue.
760 		 * When a waiting writer set the handoff flag, it may spin  in rwsem_spin_on_owner()
761 		 * on the owner as well. Once that writer acquires the lock,  in rwsem_spin_on_owner()
793  * Calculate reader-owned rwsem spinning threshold for writer
831 	 *  1) the owning writer isn't running; or  in rwsem_optimistic_spin()
856 			 * This allows a writer to steal the lock in between  in rwsem_optimistic_spin()
899 		 *    writer.  in rwsem_optimistic_spin()
904 		 * writer. Of course, if the time lag is long enough or the  in rwsem_optimistic_spin()
905 		 * new owner is not a writer or spinnable, the RT task will  in rwsem_optimistic_spin()
908 		 * If the owner is a writer, the need_resched() check is  in rwsem_optimistic_spin()
910 		 * a writer, need_resched() check needs to be done here.  in rwsem_optimistic_spin()
967  * given wake_q if the rwsem lock owner isn't a writer. If rwsem is likely
971  * This is being called from both reader and writer slow paths.
1003 	 * writer, don't attempt optimistic lock stealing if the lock is  in rwsem_down_read_slowpath()
1042 		 * by a writer, this reader can exit the slowpath and return  in rwsem_down_read_slowpath()
1163 		 * transfer. If the previous owner is a on-cpu writer and it  in rwsem_down_write_slowpath()
1365 	 * to an anonymous writer by setting the RWSEM_NONSPINNABLE bits.  in __up_write()