Uniquing

Uniquing is the mechanism in which Enterprise Objects ensures that a row in a database is associated with only one enterprise object in a given editing context in an application. The uniquing of enterprise objects limits memory usage and guarantees that the enterprise objects you work with represent the state of their associated database rows as they were last fetched into the object graph.

Without uniquing, a new enterprise object would be created every time you fetch its corresponding row, whether explicitly or through the resolution of relationships. For example, consider the case of a simple relationship between employees and a manager. Bodhi, Brent, and Ernest are represented by employee enterprise objects and Katherine is represented by a manager enterprise object that is the destination of the employee’s manager relationship.

Without uniquing, when the database row representing Bodhi is fetched, an object representing Bodhi’s manager, Katherine, is created to resolve his manager relationship. Then, when the database row representing Ernest is fetched, another object representing Katherine is created to resolve his manager relationship. If the row representing Katherine is itself explicitly fetched, yet another enterprise object representing Katherine is created. In this scenario, Katherine’s row in the database can be altered by multiple enterprise object instances, resulting in objects that represent the same row but that may contain different and conflicting data.

With uniquing, however, in a given editing context, only one object representing Katherine is ever created. All the enterprise objects in a given editing context that refer to Katherine’s enterprise object refer to the same instance—they have a single view of Katherine’s data. So within a given editing context, there is no ambiguity with regard to the data in Katherine’s enterprise object. These two scenarios are illustrated in Figure 6-4.

How does uniquing work? Objects are uniqued based on their global ID. A global ID (com.webobjects.eocontrol.EOGlobalID) is formed from an object’s primary key and its associated entity. When a row is fetched to create an object in a particular editing context, its global ID is checked against the objects already in the editing context. If a match is found, the newly fetched object isn’t added to the context.

A single enterprise object instance exists in one and only one EOEditingContext, but multiple copies of an object can exist in different editing contexts. In other words, the scope of object uniquing is a particular editing context.

Snapshotting

When an EODatabaseContext fetches objects from a database, a snapshot is recorded of the state of the fetched database row. A snapshot is a dictionary of a row’s primary keys, class properties, foreign keys used in relationships that are class properties, and the attributes of an entity that participate in optimistic locking. To learn how snapshots participate in optimistic locking, see Inside Optimistic Locking.

You can imagine that an application that fetches hundreds of rows of data builds up a large cache of snapshots. Theoretically, if enough fetches are performed, an Enterprise Objects application can contain all the contents of a database in memory. Clearly, snapshots must be managed in order to prevent this situation.

So how are snapshots cleaned up? This is the responsibility of a mechanism called snapshot reference counting. This mechanism keeps track of the enterprise objects that are associated with a particular snapshot—enterprise objects that contain data from a particular snapshot. When there are no remaining enterprise object instances associated with a particular snapshot (which Enterprise Objects determines by maintaining a list of these references), that snapshot is released.

Snapshot reference counting is handled automatically by the framework, so you don’t need to think about it.

Uniquing and Faulting

When a fault is constructed for a to-one relationship, the global ID for that fault is checked to see if the fault or its fully initialized enterprise object counterpart already exists in a given editing context. If so, that object is used to immediately resolve the relationship. This preserves the uniqueness requirement for enterprise objects by ensuring that there’s never more than one global ID representing the same row in the database. Whether that global ID represents an actual enterprise object or a fault doesn’t matter, since the data is fetched when it’s needed.

If Enterprise Objects fetches data for an object that’s already been created as a fault, that fault is fired and the enterprise object finishes initializing.

When Does Database Fetching Occur?

The first thing to understand when dealing with the issue of data freshness is to understand when Enterprise Objects uses cached data and when it fetches data from a database. In most cases, if an editing context asks an enterprise object for its data, it receives cached data unless:

• the timestamp of the snapshots of enterprise objects are older than the editing context’s timestamp

• the enterprise object has been invalidated

• the enterprise object is a fault (its data hasn’t yet been fetched)

When multiple users access the same data source by sharing an application instance, they most often share data caches. This means that one user’s data query may not actually invoke a fetch from the data source if the data requested has already been fetched by another user and so exists in the cache. A common design pattern for this situation is to provide each user with a separate snapshot cache, as discussed in Providing Separate Stacks. But even in this scenario, a user’s fetch request may not actually trigger a fetch from the database if the requested data has already been retrieved by an earlier fetch made by that user.

Distributed Change Notification?

Often, the issue of fresh data occurs when multiple users are using different application instances that all access the same data source. You want to ensure that changes one user makes are reflected in other user’s applications. Unfortunately, Enterprise Objects does not include a distributed change notification mechanism to help you with this problem (though third parties have developed solutions).

But a distributed change notification mechanism isn’t necessarily the solution. In any multiwriter database environment in which multiple users have concurrent write access to the same database, it is fundamentally impossible to guarantee fresh data. In this scenario, recovery is a better mechanism than prevention.

That is, instrumenting your applications to be resilient when update conflicts occur is a more reasonable approach than trying to prevent update conflicts altogether. Strategies for implementing resiliency are discussed in Update Strategies. That said, a mix of prevention and recovery is probably the best solution for most cases. The following sections discuss the built-in prevention mechanisms provided by the Enterprise Object frameworks.

Fetch Timestamp

Each editing context in an application includes a fetch timestamp that it uses to tell its parent object store that it wants cached data or fresh data from the database. An editing context prefers data that was fetched on or after an absolute time that is tracked by an editing context’s fetch timestamp. (Ultimately, an editing context’s parent object store decides when to perform database fetches. In the default case, an editing context’s parent object store does honor its editing context’s fetch timestamps, but this may not be the case for all object stores).

When enterprise objects are requested from an editing context, the editing context sends this request along with a fetch timestamp to its parent object store. If the requested enterprise objects have already been fetched, the parent object store finds the snapshots of those enterprise objects and compares their fetch timestamps with the fetch timestamp sent by the editing context that requested the objects.

If the timestamp of the snapshots from which the requested enterprise objects were formed is older than the editing context’s fetch timestamp, the snapshots are considered stale and fresh values for those enterprise objects are requested from the database. Otherwise, cached enterprise object values are used (these cached values are in the database context’s snapshots).

Timestamp Lag

An editing context’s fetch timestamp is set to the time of a fetch minus the default timestamp lag. The default lag is sixty minutes so the default fetch timestamp on an application’s editing contexts is one hour before a fetch occurred. So, if the timestamp of an enterprise object’s snapshots in the database context are within an hour of the fetch timestamp of the object’s editing context, a fetch returns the cached data in those snapshots rather than refetching from the database. However, if the timestamp of the snapshots in the database context are older than an hour (or older than the editing context’s fetch timestamp), the snapshots are discarded and data is refetched from the database.

A common design pattern is to set the default timestamp lag to a smaller number to encourage more refetching from the database. You can change the default timestamp lag for all the editing contexts in an application using the static method on EOEditingContext called setDefaultFetchTimestampLag. In some cases, you may want to explicitly set the fetch timestamp of a particular editing context to encourage refetching of its data. You can do this by invoking setFetchTimestamp on an editing context.

Nested editing contexts use the fetch timestamp of their parent, so applications that make heavy use of nested editing contexts may have to take additional measures to ensure fresh data.

Other Mechanisms to Ensure Freshness

Enterprise Objects provides other mechanisms to ensure the freshness of data in enterprise objects. By using the method refreshesRefetchedObjects, you force data values to be updated with fresh values from the data source when those objects are refetched. Using refreshesRefetchedObjects is described in Refreshing Cached Data.

Another, more severe mechanism to ensure fresh data is the method invalidateObjectsWithGlobalIDs on EOEditingContext, which flushes all the snapshots corresponding to the given global IDs and refetches those snapshot’s rows. It has an even more severe counterpart, invalidateAllObjects. These are rather drastic measures that you should use cautiously. See Discarding Cached Objects for more information.

As an alternative to both, you should instead consider using the method setFetchTimestampLag along with refaultAllObjects or refreshAllObjects on an editing context to update the data in enterprise objects in a given editing context.

Deferred Faulting

As described in Faulting and Relationship Resolution, Enterprise Objects uses faults to improve application performance. Fault creation is much faster than enterprise object creation and faults consume fewer resources than whole enterprise object instances. However, fault instantiation still takes time. To improve performance even further, Enterprise Objects uses deferred faults.

In an enterprise object class that can use deferred faulting, the object’s relationships are initialized as deferred faults. For a particular relationship, a single deferred fault is shared between all instances of an enterprise object class. This sharing can significantly reduce the number of faults that need to be created and usually reduces the overhead of fault creation during a fetch.

For example, consider a Listing entity that has an agent relationship. Assuming the worst case in which each Listing enterprise object has a different Agent enterprise object, without deferred faulting, a fetch of twenty Listing objects results in the creation of twenty faults for the agent relationship—a fault for each Listing. With deferred faulting, only one fault is created—a deferred fault that is shared by all the Listing objects.

If you use EOGenericRecord subclasses for your custom enterprise object classes as described in Which Enterprise Object Class?, those classes automatically use deferred faulting. If your custom enterprise object classes instead inherit from EOCustomObject and you want to enable deferred faulting in them, override the method usesDeferredFaultCreation to return true in those classes, as Listing 3-2 does. In those classes, you must also invoke willReadRelationship before accessing a relationship that might be a deferred fault.

Batch Faulting

Another advanced faulting feature is batch faulting. When a fault is fired, its data is fetched from the database. However, firing one fault has no effect on other faults—firing one fault just fetches the object or objects for the one fault. By batching fault firings together, you can more efficiently use the round trip to the database that is necessary when a single fault is fired.

For example, given an array of Employee enterprise objects, you can fetch all of the objects that are the destination of their department relationship with one round trip to the server. Without batch faulting, a round trip to the database is made to resolve each Employee’s department relationship.

There are a number of ways to implement batch faulting. You can configure batch faulting in three contexts: on entities, on relationships, and on relationships under certain circumstances.

You configure batch faulting for an entity in EOModeler in an entity’s advanced inspector, as shown in Figure 6-5. The integer you specify in the Batch Faulting Size field specifies the number of faults to fire the first time a fault is fired for any relationship in that entity. You can set this size programmatically using the method in EOEntity called setMaxNumberOfInstancesToBatchFetch.

You can also specify a batch faulting size for a particular relationship. The easiest and most common way to do this is in EOModeler using the advanced relationship inspector, which is shown in Figure 6-6. The batch size specifies the number of faults to fire when the first fault in the relationship is fired.

Finally, you can take more precise control of batch faulting by explicitly batching together faults for particular objects. When you specify the batch size in EOModeler for all of an entity’s relationships or for particular relationships, you don’t actually control which faults are fired. The method batchFetchRelationship in EODatabaseContext allows you to batch fetch all of the faults in a particular relationship. The method databaseContextShouldFetchArrayFault in EODatabaseContext.Delegate allows you to turn batch faulting on and off arbitrarily. See the API reference for EODatabaseContext and EODatabaseContext.Delegate for more details.

Prefetching

As described in Faulting and Relationship Resolution, when Enterprise Objects fetches an enterprise object, it creates faults for the object’s relationships. Each time a fault is fired, a round trip is made to the database to retrieve the fault’s data. You can batch together fault firing as described in Batch Faulting to reduce the number of round trips to the database. However, you can go even further in reducing the number of round trips to the database by prefetching all the objects in a particular relationship. Prefetching allows you to anticipate that some of an enterprise object’s relationships will be fetched and provides a mechanism to preload them; it provides a performance opportunity.

For example, consider a Listing entity that has an agent relationship. When you fetch twenty Listing objects, faults are created for each Listing’s agent relationship. When the data in the agent relationship is accessed for a particular Listing object, a fault is fired to retrieve the relationship’s data, which invokes a round trip to the database. Implementing batch faulting reduces the number of round trips but you can further reduce the number of round trips by simply prefetching all of the agent data in the database. With prefetching, when a fetch is performed for a particular entity, the objects in the relationships specified by the prefetching key paths (agent in this case) are immediately fetched.

You instrument your application for prefetching by configuring certain fetch specifications for prefetching. You can use the Prefetching pane of EOModeler’s fetch specification builder to configure it for a particular fetch specification or you can invoke setPrefetchingRelationshipKeyPaths on a fetch specification, which takes an array of strings representing the relationships to prefetch.

There are a few guidelines to consider when using prefetching. First, if memory usage is an issue for your application rather than database performance, don’t use prefetching as it consumes more memory. In fact, prefetching can consume an inordinate amount of memory depending on the size of the data set, so it’s probably more appropriate to prefetch only those relationships that have a small number of destination objects.

Second, don’t use prefetching on a fetch specification that uses a fetch limit. The prefetching hint ignores the fetch limit.

Third, don’t use prefetching when performing multiple queries that return the same records. The performance benefits of prefetching are negated by the overhead of re-creating enterprise objects of the same rows of data multiple times.

Entity Caching

Many applications have read-only entities that contain static data such as lists of states and countries, building names, or department names. Since many users of an application use the data in these entities, it makes sense to cache the data in memory to reduce the number of fetches to the database. In Enterprise Objects, you can cache an entire table in memory, thereby eliminating unnecessary fetches for the same static data by multiple users.

To enable entity caching for an entity, select the Cache In Memory option in EOModeler’s advanced entity inspector for a particular entity, as shown in Figure 6-7. You can enable this option programmatically using the method setCachesObjects on EOEntity.

When entity caching is enabled for a particular entity, the first fetch of that entity’s table causes the whole table to be fetched into memory. Clearly, this option is appropriate only for tables with a small number of rows.

An entity’s cache of objects is maintained by an EODatabaseContext. If you provide a separate access layer stack for each user as described in Providing Separate Stacks, each session has its own EODatabaseContext, so bear in mind that entity caching in this scenario may consume a lot of memory.

Raw Row Fetching

Fetch specifications provide an option to fetch raw rows. When you use raw row fetching, database rows that are fetched are not automatically transformed into enterprise object instances. There are a number of reasons why you’d want to specify raw row fetching for a particular fetch specification. These include:

• reducing memory usage when fetching large data sets

• improving application performance when fetching large data sets

• reducing the general overhead of an application instance

You can specify raw row fetching for a particular fetch specification either in EOModeler’s fetch specification builder or by invoking the method setFetchesRawRows on a fetch specification. You can more closely control which rows are fetched as raw rows using the method setRawRowKeyPaths on a fetch specification.

When you fetch raw rows, you lose many of the benefits of using full-fledged enterprise object instances such as the object graph, change notifications, and so forth. But many of the cases in which you need to fetch raw rows involve fetching large data sets that don’t need the benefits of the object graph, so this is an acceptable trade-off in light of the performance benefits of raw row fetching.

Plus, you can always instantiate an enterprise object of that row using the method faultForRawRow on an EOEditingContext.

Raw SQL Fetching

Although fetch specifications are the most common type of objects used to fetch data in Enterprise Objects applications, a lighter-weight mechanism is also provided that fetches raw rows based on an SQL expression you provide. This mechanism is provided as a method called rawRowsForSQL on the EOUtilities class. You pass to the method as arguments an editing context, a String representing the model that contains the entities on which to perform to the fetch, and a valid SQL expression. The results are returned as raw rows rather than as full-fledged enterprise objects.

Qualifiers

There are many ways to programmatically create a qualifier. One of the most common ways is to provide a format string to a qualifier. A format string is a logical expression that specifies parameters for performing a comparison. In a format string, you specify a data attribute to compare and a value with which to compare it. Enterprise Objects supports comparisons of equality, greater than, less than, greater than or equal to, less than or equal to, not equal, like, and case–insensitive like.

A format string also includes a conversion character, which specifies the data type of the value in the comparison. Table 6-3 lists the available conversion characters.

The following are examples of qualifier strings:

• agent.firstName caseInsensitiveLike %@

• bedrooms >= %d

• bathrooms <= %d AND bedrooms = %d

The following sections provide code examples that teach you how to build format strings. Many types of format strings are possible; only a few are presented here. See the API reference for com.webobjects.eocontrol.EOQualifier for more information. The following sections provide concrete code examples that show you how to build different kinds of qualifiers, after a section introducing qualifiers.

Simple String Qualifier

The code below constructs a qualifier to find the listings associated with a particular agent, based on the agent’s last name.

EOQualifier.qualifierWithQualifierFormat("agent.lastName = %s", new           NSArray(new Object[] {“Basset”, “Travers”}));

Simple Integer Qualifier

The code below constructs a qualifier to find the listings with at least 3 bedrooms.

EOQualifier.qualifierWithQualifierFormat("bedrooms >= %d", new NSArray(new Object[] {3}));

Wildcard Qualifiers

The code below constructs a qualifier to find the Listings associated with Agents whose first names begins with the letter b.

EOQualifier.qualifierWithQualifierFormat(“agent.lastName caseInsensitiveLike %@”, new           NSArray(new Object[] {“B*”}));

Compound Qualifiers

Providing a fetch specification with a single qualifier often doesn’t provide the precision you need for search criteria. Fortunately, you can easily form boolean combinations of qualifiers. These types of qualifiers, qualifiers composed of other qualifiers, are called compound qualifiers. Enterprise Objects supports AND, OR, and NOT boolean combinations of qualifiers.

This code builds a compound qualifier that combines the qualifiers constructed in Simple String Qualifier and in Wildcard Qualifiers:

EOQualifier compoundQualifier = new EOAndQualifier(new NSArray(new Object[]

     {EOQualifier.qualifierWithQualifierFormat("agent.lastName = %s", new NSArray(new

       Object[] {“Basset”, “Travers”})),

      EOQualifier.qualifierWithQualifierFormat(“agent.lastName caseInsensitiveLike %s”,

        new NSArray(new Object[] {“B*”}))}));

This creates a qualifier that searches for the criteria specified in both qualifiers.