Defining Client Compatibility

As a library upon which existing clients depend is revised, changes to it may affect the clients’ ability to use new versions of the library. The degree to which a client can use earlier or later versions of a dependent library than the one it is linked with is called client compatibility.

Some changes are minor; these include adding symbols that are unknown to clients. Other changes are major; such changes include removing a symbol, changing a symbol’s size or visibility, and changing the semantics of a function. All the symbols a library exposes to clients make up the library’s ABI (app binary interface). The library’s API (app programming interface) comprises only the functions that a library makes available to its clients. The degree to which a library’s ABI remains the same from the point of view of the clients that were developed using an earlier version of the library determines the library’s stability. Ensuring that a library’s ABI remains stable guarantees that clients can use newer versions of the library unchanged. That is, users of an app that depends on a library that’s updated regularly can see their app’s performance improve as they update the library (think of the OS X Software Update mechanism) without obtaining a new version of the app.

For example, assume an app is linked with the first version of a dynamic library and released to end users. Later, the library’s developer makes minor changes to the library and releases it. When end users install the new version of the library on their computers, the app can use the new version without requiring the end users to get an updated app binary from the developer. The app may benefit from efficiency improvements made to the library’s implementation. And, depending on how the library was written, the app might be able to take advantage of features introduced by the new version. However, these features are accessed by the app only through the API available in the first version of the library. Any interfaces introduced in the new version of the library go unused by the app.

Figure 1 illustrates the life cycle of the Draw dynamic library and one of its clients.

This list describes the versions of the library and the client:

• Draw 1.0 is the initial version of the library. It exports two functions, draw_line and draw_square.

• Client 1.0 is linked with Draw 1.0. Therefore, it can use the two symbols the library exports.

• Draw 1.1 has faster versions of draw_line and draw_square , but their semantics are unchanged, maintaining client compatibility. This is a compatible, minor revision because Client 1.0 can use Draw 1.1.

• Draw 1.2 introduces the draw_polygon function. The API of the new revision of the library is a superset of the previous version’s API. The Draw 1.1 API subset of the 1.2 version is unchanged. Therefore, Client 1.0 can use Draw 1.2. However, Client 1.0 doesn't know of the existence of draw_polygon and, therefore, it doesn’t use it. This is a minor revision because the API Client 1.0 knows about is unchanged in Draw 1.2. But this is also an incompatible revision because the API changed. Clients linked with this version of the library cannot use earlier versions.

• Client 1.1 is linked with Draw 1.2 and uses draw_polygon. Client 1.1 cannot use earlier versions of the library because it uses draw_polygon, a function that isn’t exported by those versions. However, if the library’s developer adds the weak_import attribute to the symbol’s definition, Client 1.1 would be able to use earlier versions of the library by ensuring that draw_polygon exists in its namespace before using it. If the symbol isn’t defined, the client may use other means of performing the desired task, or it may not perform the task. See Symbol Exporting Strategies for details.

• Draw 2.0 doesn’t export draw_square. This is a major revision because a symbol exported in the previous version of the library is not exported in this version. Clients linked with this version of the library cannot use earlier versions.

Clients should be able to use all the minor revisions to the library they’re linked with without relinking. In general, to use a major revision of a library, the client must be linked with the new version. The client may also need to be changed to take advantage of new symbols, to adapt its use of symbols that have been modified, or to not use symbols that are not exported by the new revision.

Specifying Version Information

The filename of a dynamic library normally contains the library’s name with the lib prefix and the .dylib extension. For example, a library called Dynamo would have the filename libDynamo.dylib. However, if a library may go through one or more revisions after it’s released, its filename must include the major version number of the revision. Clients linked with libraries whose filename includes the major version number of the revision never use a new major revision of the library because major revisions are published under different filenames. This versioning model prevents clients from using library revisions whose API is incompatible with the API known to the clients.

When you publish a dynamic library intended to have future revisions, you must disclose the library’s major version number in its filename. For example, the filename for the first version of the Draw library, introduced in Defining Client Compatibility, could be libDraw.A.dylib. The letter A specifies the major version number for the initial release. You can use any nomenclature for the major version. For example, the Draw library could also be named libDraw.1.dylib, or libDraw.I.dylib. The important thing is that the filenames of subsequent major revisions of the library have different (and preferably incremental) major version numbers. Continuing the Draw library example, a major revision to the library could be named libDraw.B.dylib , libDraw.2.dylib, or libDraw.II.dylib. Minor revisions to the library are released under the same filename used by the previous major revision.

In addition to the major version number, a library has a minor version number. The minor version number is an incremental number using the format X[.Y[.Z]] , where X is a number between 0 and 65535, and Y and Z are numbers between 0 and 255. For example, the minor version number for the first release of the Draw library could be 1.0. To set the minor version number of a dynamic library, use the clang -current_version <version_number> option.

The compatibility version number is similar to the minor version number; it’s set through the compiler -compatibility_version command-line option. The compatibility version number of a library release specifies the earliest minor version of the clients linked with that release can use. For instance, the example in Defining Client Compatibility indicates that Client 1.1 cannot use versions of the Draw library earlier than 1.2 because they don’t export the draw_polygon function. To view a library’s current and compatibility versions, use the otool -L <library> command.

Before loading a dynamic library, the dynamic loader compares the current version of the .dylib file in the user’s file system with the compatibility version of the .dylib file the client was linked with in the developer’s file system. If the current version is earlier (less) than the compatibility version, the dependent library is not loaded. Therefore, the launch process (for client apps) or the load process (for client libraries) is aborted.

Deciding What Symbols to Export

Reducing the set of symbols your library exports makes the library easy to use and easy to maintain. With a reduced symbol set, the users of your library are exposed only to the symbols that are relevant to them. And with few public symbols, you are free to make substantial changes to the internal interfaces, such as adding or removing internal symbols that do not affect clients of your library.

Global variables should never be exported. Providing uncontrolled access to a library’s global variables leaves the library open to problems caused by clients assigning inappropriate values to those variables. It’s also difficult to make changes to global variables from one version of your library to another without making newer revisions incompatible with clients that were not linked with them. One of the main features of dynamic libraries is the fact that, when implemented correctly, clients can use newer versions of them without relinking. If clients need to access a value stored in a global variable, your library should export accessor functions but not the global variable itself. Adhering to this guideline allows library developers to change the definitions of global variables between versions of the library, without introducing incompatible revisions.

If your library needs the functionality implemented by functions it exports, you should consider implementing internal versions of the functions, adding wrapper functions to them, and exporting the wrappers. For example, your library may have a function whose arguments must be validated, but you’re certain that the library always provides valid values when invoking the function. The internal version of the function could be optimized by removing validation code from it, making internal use more efficient. The validation code can then be placed in the wrapper function, maintaining the validation process for clients. In addition, you can further change the internal implementation of the function to include more parameters, for example, while maintaining the external version the same.

Having wrapper functions call internal versions reduces the performance of an app, especially if the function is called repeatedly by clients. However, the advantages of flexible maintenance for you and a stable interface for your clients greatly outweigh this negligible performance impact.

Naming Exported Symbols

The dynamic loader doesn’t detect naming conflicts between the symbols exported by the dynamic libraries it loads. When a client contains a reference to a symbol that two or more of its dependent libraries export, the dynamic loader binds the reference to the first dependent library that exports the symbol in the client’s dependent library list. The dependent library list is a list of the client’s dependent libraries in the order they were specified when the client was linked with them. Also, when the dlsym(3) OS X Developer Tools Manual Page function is invoked, the dynamic loader returns the address of the first symbol it finds in the specified scope (global, local, or next) with a matching name. For details on symbol-search scope, see Using Symbols.

To ensure that your library’s clients always have access to the symbols your library exports, the symbols must have unique names in a process’s namespace. One way is for apps to use two-level namespaces. Another is to add prefixes to every exported symbol. This is the convention used by most of the OS X frameworks, such as Carbon and Cocoa. For more information on two-level namespace, see Executing Mach-O Files in Mach-O Programming Topics.

Symbol Exporting Strategies

After you have identified the symbols you want to expose to your library’s users, you must devise a strategy for exporting them or for not exporting the rest of the symbols. This process is also known as setting the visibility of the symbols—that is, whether they are accessible to clients. Public or exported symbols are accessible to clients; private, hidden, or unexported symbols are not accessible to clients. In OS X, there are several ways of specifying the visibility of a library’s symbols:

• The static storage class: This is the easiest way to indicate that you don’t want to export a symbol.

• The exported symbols list or the unexported symbols list: The list is a file with the names of symbols to export or a list of symbols to keep private. The symbol names must include the underscore (_) prefix. You can use only one type of list when generating the dynamic library file.

• The visibility attribute: You place this attribute in the definition of symbols in implementation files to set the visibility of symbols individually. It gives you more granular control over which symbols are public or private.

• The compiler -fvisibility command-line option: This option specifies at compilation time the visibility of symbols with unspecified visibility in implementation files. This option, combined with the visibility attribute, is the most safe and convenient way of identifying public symbols.

• The weak_import attribute: Placing this attribute in the declaration of a symbol in a header file tells the compiler to generate a weak reference to the symbol. This feature is called weak linking; symbols with the weak_import attribute are called weakly linked symbols. With weak linking, clients do not fail to launch when the version of the dependent library found at launch time or load time doesn’t export a weakly linked symbol referenced by the client. It’s important to place the weak_import attribute in the header files that the source files of the library’s clients use, so that the client developers know that they must ensure the existence of the symbol before using it. Otherwise, the client would crash or function incorrectly when it attempts to use the symbol. See Using Weakly Linked Symbols for further details on weakly linked symbols. For more information on symbol definitions, see Executing Mach-O Files in Mach-O Programming Topics.

• The compiler -weak_library command-line option: This option tells the compiler to treat all the library’s exported symbols as weakly linked symbols.

To illustrate how to set the visibility of a library’s symbols, let’s start with a dynamic library that allows its clients to set a value kept in a global variable in the library, and to retrieve the value. Listing 1 shows the code that makes up the library.

Listing 1  A simple dynamic library

/* File: Person.h */

char* name(void);

void set_name(char* name);

/* File: Person.c */

#include "Person.h"

#include <string.h>

char _person_name[30] = {'\0'};

char* name(void) {

    return _person_name;

}

void _set_name(char* name) {

   strcpy(_person_name, name);

}

void set_name(char* name) {

    if (name == NULL) {

        _set_name("");

    }

    else {

        _set_name(name);

    }

}

The intent of the library’s developer is to provide clients the ability to set the value of _person_name with the set_name function and to let them obtain the value of the variable with the name function. However, the library exports more than the name and set_name functions, as shown by the output of the nm command-line tool:

% clang -dynamiclib Person.c -o libPerson.dylib

% nm -gm libPerson.dylib

                 (undefined) external ___strcpy_chk (from libSystem)

0000000000001020 (__DATA,__common) external __person_name     // Inadvertently exported

0000000000000e80 (__TEXT,__text) external __set_name          // Inadvertently exported

0000000000000e70 (__TEXT,__text) external _name

0000000000000ec0 (__TEXT,__text) external _set_name

                 (undefined) external dyld_stub_binder (from libSystem)

Note that the _person_name global variable and the _set_name function are exported along with the name and set_name functions. There are many options to remove _person_name and _set_name from the symbols exported by the library. This section explores a few.

The first option is to add the static storage class to the definition of _person_name and _set_name in Person.c , as shown in Listing 2.

Listing 2  Person module hiding a symbol with the static storage class

/* File: Person.c */

#include "Person.h"

#include <string.h>

static char _person_name[30] = {'\0'};        // Added 'static' storage class

char* name(void) {

    return _person_name;

}

static void _set_name(char* name) {           // Added 'static' storage class

   strcpy(_person_name, name);

}

void set_name(char* name) {

    if (name == NULL) {

        _set_name("");

    }

    else {

        _set_name(name);

    }

}

Now, the nm output, looks like this:

                 (undefined) external ___strcpy_chk (from libSystem)

0000000000000e80 (__TEXT,__text) external _name

0000000000000e90 (__TEXT,__text) external _set_name

                 (undefined) external dyld_stub_binder (from libSystem)

This means that the library exports only name and set_name. Actually, the library also exports some undefined symbols, including strcpy. They are references to symbols the library obtains from its dependent libraries.

The problem with this approach is that it hides the internal _set_name function from other modules in the library. If the library’s developer trusts that any internal call to _set_name doesn’t need to be validated but wants to validate all client calls, the symbol must be visible to other modules within the library but not to the library’s client. Therefore, the static storage class is not appropriate to hide symbols from the client but disclose them to all the library’s modules.

A second option for exposing only the symbols intended for client use is to have an exported symbols file that lists the symbols to export; all other symbols are hidden. Listing 3 shows the export_list file.

Listing 3  File listing the names of the symbols to export

# File: export_list

_name

_set_name

To compile the library, you use the clang -exported_symbols_list option to specify the file containing the names of the symbols to export, as shown here:

clang -dynamiclib Person.c -exported_symbols_list export_list -o libPerson.dylib

The third and most convenient option for exposing only name and set_name is to set the visibility attribute in their implementations to "default" and set the -fvisibility compiler command-line option to hidden when compiling the library’s source files. Listing 4 shows how the Person.c file looks after setting the visibility attribute for the symbols to be exported.

Listing 4  Person module using visibility attribute to export symbols

/* File: Person.c */

#include "Person.h"

#include <string.h>

// Symbolic name for visibility("default") attribute.

#define EXPORT __attribute__((visibility("default")))

char _person_name[30] = {'\0'};

EXPORT                        // Symbol to export

char* name(void) {

    return _person_name;

}

void _set_name(char* name) {

   strcpy(_person_name, name);

}

EXPORT                        // Symbol to export

void set_name(char* name) {

    if (name == NULL) {

        _set_name("");

    }

    else {

        _set_name(name);

    }

}

The library would then be compiled using the following command:

% clang -dynamiclib Person.c -fvisibility=hidden -o libPerson.dylib

The -fvisibility=hidden command-line option tells the compiler to set the visibility of any symbols without a visibility attribute to hidden, thereby hiding them from the library’s clients. For details on the visibility attribute and the -fvisibility command-line option, see http://gcc.gnu.org/onlinedocs/gcc and the clang man page.

Following these symbol-exporting guidelines ensures that libraries export only the symbols you want to make available to your clients, simplifying the use of the library by its clients and facilitating its maintenance by its developers. The document How to Write Shared Libraries provides an in-depth analysis of symbol exporting strategies. This document is available at http://people.redhat.com/drepper/dsohowto.pdf.

Exporting C++ Symbols

C++ uses name mangling to encode size and type information in a symbol name. Name mangling in C++ makes exporting symbols in a standard way across different platforms impossible. When a dynamic library is compiled, each symbol is renamed to include that information, but the encoding used is not standard between platforms. The dynamic loader uses string matching to locate symbols at runtime; the name of the symbol searched must match exactly the name of the target. When the symbol names have been mangled, the dynamic loader has no way of knowing how the name of the symbol it’s searching for has been encoded.

To export nonmember symbols from a C++–based dynamic library so that the dynamic loader can find it, you must add the extern "C" directive to the symbols’ declarations. This keyword tells the compiler not to mangle the symbol name. For example, the following declaration makes the NewPerson function available to the library’s clients:

extern "C" Person* NewPerson(void);

Without this directive, the function name could be changed to _Z9NewPersonv , which would make it impossible for the dynamic loader to find the NewPerson symbol at runtime.

The only nonmember functions that must be exported are constructors and destructors, especially in dynamic libraries that can be used by clients as dependent libraries instead of runtime-loaded libraries. This is because clients must have access to a class’s constructors and destructors so that they can use the new and delete operators on the class.

Defining C++ Class Interfaces

A dynamic library should always publish its public interface to clients through header files. (Although clients can use dynamic libraries without their header files, doing so is very difficult and is prone to error.) The header file for a class that’s available to clients must include the declarations of its public methods. Dynamic libraries that make a class available to its clients must include the virtual keyword in the declaration of all the class’s methods, except for its constructors and destructors. For example, Listing 9 shows the declaration for the Person class.

Listing 9  Declaration for the Person class

class Person {

    private:

        char _person_name[30];

    public:

        Person();

        virtual void set_name(char person_name[]);

        virtual char* name();

};

Creating and Destroying C++ Objects

When a C++–based dynamic library is a dependent library of its client, the client can use the new and delete operators to create and destroy instances of a class the library defines. However, clients that open a C++–based library at runtime through dlopen(3) OS X Developer Tools Manual Page do not have access to the library’s constructors because the constructors are exported with their names mangled, preventing the dynamic loader from locating them. See Listing 11 for details about name mangling.

Clients that load a library at runtime to use a class must have a way to create and destroy a class’s objects without using the new and delete operators. A library provides this functionality to its clients by exporting at least two class factory functions. Class factory functions create and destroy objects of a class on behalf of a class’s user. Creator functions create an object of the class and return a pointer to it. Destructor functions dispose of an object created by a creator function for the same class. In addition to the factory functions, the library must define a data type for each exported factory function.

For example, Listing 10 shows the header and implementation files of the Person class, exported by the Person library. The header file includes the declaration and type definition of a pair of factory functions, NewPerson, and DeletePerson:

Listing 10  Interface and implementation of a C++ class in a dynamic library

/* File: Person.h */

class Person {

    private:

        char _person_name[30];

    public:

        Person();

        virtual void set_name(char person_name[]);

        virtual char* name();

};

// Constructor function and function type.

extern "C" Person *NewPerson(void);

typedef Person *Person_creator(void);

// Destructor function and function type.

extern "C" void DeletePerson(Person *person);

typedef void Person_disposer(Person *);

/* File: Person.cpp */

include <iostream>

#include "Person.h"

#define EXPORT __attribute__((visibility("default")))

EXPORT

Person::Person() {

    char default_name[] = "<no value>";

    this->set_name(default_name);

}

EXPORT

Person* NewPerson(void) {

    return new Person;

}

EXPORT

void DeletePerson(Person* person) {

    delete person;

}

void Person::set_name(char name[]) {

    strcpy(_person_name, name);

}

char* Person::name(void) {

    return _person_name;

}

Listing 11 shows how a client might use the Person library.

Listing 11  Client using a C++ class implemented in a runtime-loaded library

/* File: Client.cpp */

#include <iostream>

#include <dlfcn.h>

#include "Person.h"

int main() {

    using std::cout;

    using std::cerr;

    // Open the library.

    void* lib_handle = dlopen("./libPerson.dylib", RTLD_LOCAL);

    if (!lib_handle) {

        cerr << "[" << __FILE__ << "] main: Unable to open library: "

             << dlerror() << "\n";

        exit(EXIT_FAILURE);

    }

    // Get the NewPerson function.

    Person_creator* NewPerson = (Person_creator*)dlsym(lib_handle, "NewPerson");

    if (!NewPerson) {

        cerr << "[" << __FILE__ << "] main: Unable to find NewPerson method: "

             << dlerror() << "\n";

        exit(EXIT_FAILURE);

    }

    // Get the DeletePerson function.

    Person_disposer* DeletePerson =

        (Person_disposer*)dlsym(lib_handle, "DeletePerson");

    if (!DeletePerson) {

        cerr << "[" << __FILE__

             << "] main: Unable to find DeletePerson method: "

             << dlerror() << "\n";

        exit(EXIT_FAILURE);

    }

    // Create Person object.

    Person* person = (Person*)NewPerson();

    // Use Person object.

    cout << "[" << __FILE__ << "] person->name() = " << person->name() << "\n";

    char new_name[] = "Floriane";

    person->set_name(new_name);

    cout << "[" << __FILE__ << "] person->name() = " << person->name() << "\n";

    // Destroy Person object.

    DeletePerson(person);

    // Close the library.

    if (dlclose(lib_handle) != 0) {

        cerr << "[" << __FILE__ << "] main: Unable to close library: "

             << dlerror() << "\n";

    }

    return 0;

}

The following commands compile the library and the client program:

% clang++ -dynamiclib Person.cpp -fvisibility=hidden -o libPerson.dylib

% clang++ Client.cpp -o client

For further details on how clients use C++ classes implemented in dynamic libraries, see Using C++ Classes.

Defining Class and Category Interfaces

Because client developers generally don’t have access to the implementation of Objective-C classes and categories defined in dynamic libraries, library developers must publish the public interfaces of classes and categories as protocols in header files. Client developers compile their products using those header files and are able to instantiate the classes correctly by adding the necessary protocol names to variable definitions. Listing 12 shows the header and implementation files of the Person class in an Objective-C–based library. Listing 13 shows the header and implementation files of the Titling category in the same library, which adds the -setTitle method to the Person class.

Listing 12  Header and implementation files of the Person class

/* File: Person.h */

#import <Foundation/Foundation.h>

@protocol Person

- (void)setName:(NSString*)name;

- (NSString*)name;

@end

@interface Person : NSObject <Person> {

    @private

    NSString* _person_name;

}

@end

/* File: Person.m */

#import <Foundation/Foundation.h>

#import "Person.h"

@implementation Person

- (id)init {

    if (self = [super init]) {

        _person_name = @"";

    }

    return self;

}

- (void)setName:(NSString*)name {

    _person_name = name;

}

- (NSString*)name {

    return _person_name;

}

@end

Listing 13  Header and implementation files of the Titling category to the Person class

/* File: Titling.h */

#import <Foundation/Foundation.h>

#import "Person.h"

@protocol Titling

- (void)setTitle:(NSString*)title;

@end

@interface Person (Titling) <Titling>

@end

/* File: Titling.m */

#import <Foundation/Foundation.h>

#import "Titling.h"

@implementation Person (Titling)

- (void)setTitle:(NSString*)title {

    [self setName:[[title stringByAppendingString:@" "]

        stringByAppendingString:[self name]]];

}

@end

Listing 14 shows how a client might use the library.

Listing 14  Client using the Person library

/* File: Client.m */

#import <Foundation/Foundation.h>

#import <objc/runtime.h>

#import <dlfcn.h>

#import "Person.h"

#import "Titling.h"

int main() {

    @autoreleasepool {

        // Open the library.

        void* lib_handle = dlopen("./libPerson.dylib", RTLD_LOCAL);

        if (!lib_handle) {

            NSLog(@"[%s] main: Unable to open library: %s\n",

            __FILE__, dlerror());

            exit(EXIT_FAILURE);

        }

        // Get the Person class (required with runtime-loaded libraries).

        Class Person_class = objc_getClass("Person");

        if (!Person_class) {

            NSLog(@"[%s] main: Unable to get Person class", __FILE__);

            exit(EXIT_FAILURE);

        }

        // Create an instance of Person.

        NSLog(@"[%s] main: Instantiating Person_class", __FILE__);

        NSObject<Person,Titling>* person = [Person_class new];

        // Use person.

        [person setName:@"Perrine LeVan"];

        [person setTitle:@"Ms."];

        NSLog(@"[%s] main: [person name] = %@", __FILE__, [person name]);

        // Close the library.

        if (dlclose(lib_handle) != 0) {

            NSLog(@"[%s] Unable to close library: %s\n",

                __FILE__, dlerror());

            exit(EXIT_FAILURE);

        }

    }

    return(EXIT_SUCCESS);

}

The following commands compile the library and the client program:

clang -framework Foundation -dynamiclib Person.m Titling.m -o libPerson.dylib

clang -framework Foundation Client.m -o client

Initializing Objective-C Classes

Objective-C–based dynamic libraries provide several initialization facilities for modules, classes, and categories. The following list describes those facilities in the order they are executed.

1. The +load method: Initializes resources needed by a class or a category. The Objective-C runtime sends the load message to every class a library implements; it then sends the load message to every category the library implements. The order in which sibling classes are sent the load message is undetermined. Implement the +load method to initialize resources needed by a class or category. Note that there’s no corresponding “unload” method.

2. Module initializers: Initializes a module. The dynamic loader calls all the initializer functions (defined with the constructor attribute) in each of a library’s modules. See Module Initializers and Finalizers for more information on module initializers.

3. The +initialize method: Initializes resources needed by instances of a class before any instances are created. The Objective-C runtime sends the initialize message to a class just before creating an instance of the class. Note that there’s no corresponding “finalize” message sent to a class when the library is unloaded or the process terminates.

Creating Aliases to a Class

When you rename a class in a revision of a dynamic library, you can reduce the adoption burden to client developers by adding an alias to the new name in the library’s header file. This practice allows client developers to release clients that take advantage of the new version of the library quickly. Client developers can later update references to the class at their leisure.