Spinlocks

A spinlock is the simplest type of lock. In a system with a test-and-set instruction or the equivalent, the code looks something like this:

while (test_and_set(bit) != 0);

In other words, until the lock is available, it simply “spins” in a tight loop that keeps checking the lock until the thread’s time quantum expires and the next thread begins to execute. Since the entire time quantum for the first thread must complete before the next thread can execute and (possibly) release the lock, a spinlock is very wasteful of CPU time, and should be used only in places where a mutex cannot be used, such as in a hardware exception handler or low-level interrupt handler.

Note that a thread may not block while holding a spinlock, because that could cause deadlock. Further, preemption is disabled on a given processor while a spinlock is held.

There are three basic types of spinlocks available in OS X: lck_spin_t (which supersedes simple_lock_t), usimple_lock_t, and hw_lock_t. You are strongly encouraged to not use hw_lock_t; it is only mentioned for the sake of completeness. Of these, only lck_spin_t is accessible from kernel extensions.

The u in usimple stands for uniprocessor, because they are the only spinlocks that provide actual locking on uniprocessor systems. Traditional simple locks, by contrast, disable preemption but do not spin on uniprocessor systems. Note that in most contexts, it is not useful to spin on a uniprocessor system, and thus you usually only need simple locks. Use of usimple locks is permissible for synchronization between thread context and interrupt context or between a uniprocessor and an intelligent device. However, in most cases, a mutex is a better choice.

The spinlock functions accessible to kernel extensions consist of the following:

 extern lck_spin_t     *lck_spin_alloc_init(

          lck_grp_t     *grp,

          lck_attr_t     *attr);

 extern void lck_spin_init(

          lck_spin_t     *lck,

          lck_grp_t     *grp,

          lck_attr_t     *attr);

 extern void lck_spin_lock(

          lck_spin_t     *lck);

 extern void lck_spin_unlock(

          lck_spin_t     *lck);

 extern void lck_spin_destroy(

          lck_spin_t     *lck,

          lck_grp_t     *grp);

 extern void lck_spin_free(

          lck_spin_t     *lck,

          lck_grp_t     *grp);

 extern wait_result_t lck_spin_sleep(

          lck_spin_t     *lck,

          lck_sleep_action_t     lck_sleep_action,

          event_t     event,

          wait_interrupt_t     interruptible);

 extern wait_result_t lck_spin_sleep_deadline(

          lck_spin_t     *lck,

          lck_sleep_action_t     lck_sleep_action,

          event_t     event,

          wait_interrupt_t     interruptible,

          uint64_t     deadline);

Prototypes for these locks can be found in <kern/locks.h>.

The arguments to these functions are described in detail in Using Lock Functions.

Mutexes

A mutex, mutex lock, or sleep lock, is similar to a spinlock, except that instead of constantly polling, it places itself on a queue of threads waiting for the lock, then yields the remainder of its time quantum. It does not execute again until the thread holding the lock wakes it (or in some user space variations, until an asynchronous signal arrives).

Mutexes are more efficient than spinlocks for most purposes. However, they are less efficient in multiprocessing environments where the expected lock-holding time is relatively short. If the average time is relatively short but occasionally long, spin/sleep locks may be a better choice. Although OS X does not support spin/sleep locks in the kernel, they can be easily implemented on top of existing locking primitives. If your code performance improves as a result of using such locks, however, you should probably look for ways to restructure your code, such as using more than one lock or moving to read-write locks, depending on the nature of the code in question. See Spin/Sleep Locks for more information.

Because mutexes are based on blocking, they can only be used in places where blocking is allowed. For this reason, mutexes cannot be used in the context of interrupt handlers. Interrupt handlers are not allowed to block because interrupts are disabled for the duration of an interrupt handler, and thus, if an interrupt handler blocked, it would prevent the scheduler from receiving timer interrupts, which would prevent any other thread from executing, resulting in deadlock.

For a similar reason, it is not reasonable to block within the scheduler. Also, blocking within the VM system can easily lead to deadlock if the lock you are waiting for is held by a task that is paged out.

However, unlike simple locks, it is permissible to block while holding a mutex. This would occur, for example, if you took one lock, then tried to take another, but the second lock was being held by another thread. However, this is generally not recommended unless you carefully scrutinize all uses of that mutex for possible circular waits, as it can result in deadlock. You can avoid this by always taking locks in a certain order.

In general, blocking while holding a mutex specific to your code is fine as long as you wrote your code correctly, but blocking while holding a more global mutex is probably not, since you may not be able to guarantee that other developers’ code obeys the same ordering rules.

A Mach mutex is of type mutex_t. The functions that operate on mutexes include:

lck_mtx_t           *lck_mtx_alloc_init(lck_grp_t       *grp,

                                        lck_attr_t      *attr);

extern void         lck_mtx_init(       lck_mtx_t       *lck,

                                        lck_grp_t       *grp,

                                        lck_attr_t      *attr);

extern void         lck_mtx_lock(   lck_mtx_t           *lck);

extern void         lck_mtx_unlock( lck_mtx_t           *lck);

extern void         lck_mtx_destroy(lck_mtx_t           *lck,

                                    lck_grp_t           *grp);

extern void         lck_mtx_free(   lck_mtx_t           *lck,

                                    lck_grp_t           *grp);

extern wait_result_tlck_mtx_sleep(  lck_mtx_t           *lck,

                                    lck_sleep_action_t  lck_sleep_action,

                                    event_t             event,

                                    wait_interrupt_t    interruptible);

extern wait_result_tlck_mtx_sleep_deadline(

                                    lck_mtx_t           *lck,

                                    lck_sleep_action_t  lck_sleep_action,

                                    event_t             event,

                                    wait_interrupt_t    interruptible,

                                    uint64_t            deadline);

extern void         lck_mtx_assert( lck_mtx_t           *lck,

                                    unsigned int        type);

as described in <kern/locks.h>.

The arguments to these functions are described in detail in Using Lock Functions.

Read-Write Locks

Read-write locks (also called shared-exclusive locks) are somewhat different from traditional locks in that they are not always exclusive locks. A read-write lock is useful when shared data can be reasonably read concurrently by multiple threads except while a thread is modifying the data. Read-write locks can dramatically improve performance if the majority of operations on the shared data are in the form of reads (since it allows concurrency), while having negligible impact in the case of multiple writes.

A read-write lock allows this sharing by enforcing the following constraints:

• Multiple readers can hold the lock at any time.

• Only one writer can hold the lock at any given time.

• A writer must block until all readers have released the lock before obtaining the lock for writing.

• Readers arriving while a writer is waiting to acquire the lock will block until after the writer has obtained and released the lock.

The first constraint allows read sharing. The second constraint prevents write sharing. The third prevents read-write sharing, and the fourth prevents starvation of the writer by a steady stream of incoming readers.

Mach read-write locks also provide the ability for a reader to become a writer and vice-versa. In locking terminology, an upgrade is when a reader becomes a writer, and a downgrade is when a writer becomes a reader. To prevent deadlock, some additional constraints must be added for upgrades and downgrades:

• Upgrades are favored over writers.

• The second and subsequent concurrent upgrades will fail, causing that thread’s read lock to be released.

The first constraint is necessary because the reader requesting an upgrade is holding a read lock, and the writer would not be able to obtain a write lock until the reader releases its read lock. In this case, the reader and writer would wait for each other forever. The second constraint is necessary to prevent the deadlock that would occur if two readers wait for the other to release its read lock so that an upgrade can occur.

The functions that operate on read-write locks are:

extern lck_rw_t *lck_rw_alloc_init(

            lck_grp_t               *grp,

            lck_attr_t              *attr);

extern void lck_rw_init(

            lck_rw_t                *lck,

            lck_grp_t               *grp,

            lck_attr_t              *attr);

extern void lck_rw_lock(

            lck_rw_t                *lck,

            lck_rw_type_t   lck_rw_type);

extern void lck_rw_unlock(

            lck_rw_t                *lck,

            lck_rw_type_t   lck_rw_type);

extern void lck_rw_lock_shared(

            lck_rw_t                *lck);

extern void lck_rw_unlock_shared(

            lck_rw_t                *lck);

extern void lck_rw_lock_exclusive(

            lck_rw_t                *lck);

extern void lck_rw_unlock_exclusive(

            lck_rw_t                *lck);

extern void lck_rw_destroy(

            lck_rw_t                *lck,

            lck_grp_t               *grp);

extern void lck_rw_free(

            lck_rw_t                *lck,

            lck_grp_t               *grp);

extern wait_result_t lck_rw_sleep(

            lck_rw_t                        *lck,

            lck_sleep_action_t      lck_sleep_action,

            event_t                         event,

            wait_interrupt_t        interruptible);

extern wait_result_t lck_rw_sleep_deadline(

            lck_rw_t                        *lck,

            lck_sleep_action_t      lck_sleep_action,

            event_t                         event,

            wait_interrupt_t        interruptible,

            uint64_t                        deadline);

This is a more complex interface than that of the other locking mechanisms, and actually is the interface upon which the other locks are built.

The functions lck_rw_lock and lck_rw_unlock lock and unlock a lock as either shared (read) or exclusive (write), depending on the value of lck_rw_type., which can contain either LCK_RW_TYPE_SHARED or LCK_RW_TYPE_EXCLUSIVE. You should always be careful when using these functions, as unlocking a lock held in shared mode using an exclusive call or vice-versa will lead to undefined results.

The arguments to these functions are described in detail in Using Lock Functions.

Spin/Sleep Locks

Spin/sleep locks are not implemented in the OS X kernel. However, they can be easily implemented on top of existing locks if desired.

For short waits on multiprocessor systems, the amount of time spent in the context switch can be greater than the amount of time spent spinning. When the time spent spinning while waiting for the lock becomes greater than the context switch overhead, however, mutexes become more efficient. For this reason, if there is a large degree of variation in wait time on a highly contended lock, spin/sleep locks may be more efficient than traditional spinlocks or mutexes.

Ideally, a program should be written in such a way that the time spent holding a lock is always about the same, and the choice of locking is clear. However, in some cases, this is not practical for a highly contended lock. In those cases, you may consider using spin/sleep locks.

The basic principle of spin/sleep locks is simple. A thread takes the lock if it is available. If the lock is not available, the thread may enter a spin cycle. After a certain period of time (usually a fraction of a time quantum or a small number of time quanta), the spin routine’s time-out is reached, and it returns failure. At that point, the lock places the waiting thread on a queue and puts it to sleep.

In other variations on this design, spin/sleep locks determine whether to spin or sleep according to whether the lock-holding thread is currently on another processor (or is about to be).

For short wait periods on multiprocessor computers, the spin/sleep lock is more efficient than a mutex, and roughly as efficient as a standard spinlock. For longer wait periods, the spin/sleep lock is significantly more efficient than the spinlock and only slightly less efficient than a mutex. There is a period near the transition between spinning and sleeping in which the spin/sleep lock may behave significantly worse than either of the basic lock types, however. Thus, spin/sleep locks should not be used unless a lock is heavily contended and has widely varying hold times. When possible, you should rewrite the code to avoid such designs.

Using Lock Functions

While most of the locking functions are straightforward, there are a few details related to allocating, deallocating, and sleeping on locks that require additional explanation. As the syntax of these functions is identical across all of the lock types, this section explains only the usage for spinlocks. Extending this to other lock types is left as a (trivial) exercise for the reader.

The first thing you must do when allocating locks is to allocate a lock group and a lock attribute set. Lock groups are used to name locks for debugging purposes and to group locks by function for general understandability. Lock attribute sets allow you to set flags that alter the behavior of a lock.

The following code illustrates how to allocate an attribute structure and a lock group structure for a lock. In this case, a spinlock is used, but with the exception of the lock allocation itself, the process is the same for other lock types.

Listing 17-1  Allocating lock attributes and groups (lifted liberally from kern_time.c)

lck_grp_attr_t *tz_slock_grp_attr;

lck_grp_t *tz_slock_grp;

lck_attr_t *tz_slock_attr;

lck_spin_t *tz_slock;

/* allocate lock group attribute and group */

tz_slock_grp_attr = lck_grp_attr_alloc_init();

lck_grp_attr_setstat(tz_slock_grp_attr);

tz_slock_grp =  lck_grp_alloc_init("tzlock", tz_slock_grp_attr);

/* Allocate lock attribute */

tz_slock_attr = lck_attr_alloc_init();

//lck_attr_setdebug(tz_slock_attr); // set the debug flag

//lck_attr_setdefault(tz_slock_attr); // clear the debug flag

/* Allocate the spin lock */

tz_slock = lck_spin_alloc_init(tz_slock_grp, tz_slock_attr);

The first argument to the lock initializer, of type lck_grp_t, is a lock group. This is used for debugging purposes, including lock contention profiling. The details of lock tracing are beyond the scope of this document, however, every lock must belong to a group (even if that group contains only one lock).

The second argument to the lock initializer, of type lck_attr_t, contains attributes for the lock. Currently, the only attribute available is lock debugging. This attribute can be set using lck_attr_setdebug and cleared with lck_attr_setdefault.

To dispose of a lock, you simply call the matching free functions. For example:

lck_spin_free(tz_slock, tz_slock_grp);

lck_attr_free(tz_slock_attr);

lck_grp_free(tz_slock_grp);

lck_grp_attr_free(tz_slock_grp_attr);

The other two interesting functions are lck_spin_sleep and lck_spin_sleep_deadline. These functions release a spinlock and sleep until an event occurs, then wake. The latter includes a timeout, at which point it will wake even if the event has not occurred.

extern wait_result_t lck_spin_sleep(

                lck_rspin_t         *lck,

                lck_sleep_action_t  lck_sleep_action,

                event_t             event,

                wait_interrupt_t    interruptible);

extern wait_result_t lck_spin_sleep_deadline(

                lck_spin_t          *lck,

                lck_sleep_action_t  lck_sleep_action,

                event_t             event,

                wait_interrupt_t    interruptible,

                uint64_t            deadline);

The parameter lck_sleep_action controls whether the lock will be reclaimed after sleeping prior to this function returning. The valid options are:

LCK_SLEEP_DEFAULT

Release the lock while waiting for the event, then reclaim it. Read-write locks are held in the same mode as they were originally held.

LCK_SLEEP_UNLOCK

Release the lock and return with the lock unheld.

LCK_SLEEP_SHARED

Reclaim the lock in shared mode (read-write locks only).

LCK_SLEEP_EXCLUSIVE

Reclaim the lock in exclusive mode (read-write locks only).

The event parameter can be any arbitrary integer, but it must be unique across the system. To ensure uniqueness, a common programming practice is to use the address of a global variable (often the one containing a lock) as the event value. For more information on these events, see Event and Timer Waits.

The parameter interruptible indicates whether the scheduler should allow the wait to be interrupted by asynchronous signals. If this is false, any false wakes will result in the process going immediately back to sleep (with the exception of a timer expiration signal, which will still wake lck_spin_sleep_deadline).