Retired Document
Important: Apple recommends that developers explore QTKit and Core Video for new development in this technology area. See QTKit Framework Reference and Core Video Programming Guide for more information.
Creating Advanced Interactive Movies
QuickTime 4.1 introduced a set of new features for authors and tool developers that allow for the creation of even more advanced, interactive movies. These features included
the introduction of embedded movies
the addition of new wired actions and events
the addition of new ways to communicate between a wired movie and JavaScript in a Web browser
The current version of QuickTime also introduces new features for manipulating text tracks in QuickTime movies, along with other capabilities.
Embedded Movies
Embedded movies are implemented through the track type––the movie track, which is discussed in the section Movie Track and Movie Wired Actions.
Creating New Types of QuickTime Movies
Because embedded movies can have independent clocks, new types of movies can be created.
For example, you can create a movie containing animated characters that are watching a video. This movie could contain an animation track, perhaps a sprite or Flash track, and an embedded movie track for the video content. In this example, the root movie’s time base would control the animation, but the video’s rate could be controlled independently from the animations’ rate. Wired actions could be sent to the embedded movie when a user clicks on buttons in the animation track. The wired actions could play, pause, and fast forward the video, or switch to a new one.
Another way you can take advantage of independent time bases is to allow long audio tracks to be triggered interactively. For example, if you create a game with sound effects and background music that need to be played back at times defined by events that occur in the game, you can use an embedded movie for each audio track. The advantage of using an embedded movie instead of a music track with a custom sound is that the entire sound does not need to be loaded into memory, so it is appropriate for longer sounds. The disadvantage is that you may not control it similar to a MIDI instrument.
Using Embedded Movies
One way to use embedded movies is to break projects into components, allowing portions to be reused in other projects, and simplifying the authoring process. In QuickTime, this technique could be used in a Web browser using external movie-to-movie communication. In QuickTime, a single movie can be created that is playable in the QuickTime Player or any application that plays QuickTime movies.
For example, an interactive movie could contain three elements, as illustrated in Figure 7-1:
A control strip with custom controls for audio and scene navigation
An area used to display text
An area used for interactive animation
By encapsulating these three elements each in a separate movie, you can reload only portions that are necessary to reload. For example, the control strip may persist throughout the entire experience, while the animation area may be replaced once per scene and the displayed text changed several times per scene. By implementing each element as a movie and composing them together using movie tracks in a container movie, you can minimize reload time and memory usage, and even update the source movies on your Web server as you improve or change them.
Dynamically Loading Embedded Movies From URLs
A movie track maintains a list of movies that may be loaded and played within the track. The movie track plays only one movie from the list at a given time. This list is initialized from data in a movie track sample, but the list may be augmented at runtime. Each entry in the list is identified by a unique ID, and contains a data reference to a movie. There are two wired actions, which allow you to add a new URL data reference to this list, and to load and play a movie from this list.
Dynamically loading a movie into a movie track is similar to loading a URL into a frame of a Web page. This dynamic loading allows for movies to manage memory efficiently by loading QuickTime playable content as it is needed. In addition, it allows for content which is generated on a Web server to be loaded into a movie; this content could even be created based upon information that a wired movie sends to the server using the GotoURL
wired action.
Triggering Wired Actions When an Embedded Movie is Loaded
A MovieLoaded
QuickTime event allows wired actions to be executed when an embedded movie is loaded. There are two places to store these actions. The movie that is being loaded may store actions in its movie properties atom container, and the movie track may store these actions in its samples.
These wired actions can be used to examine the current state of the parent movie, and to make changes both to the movie being loaded and to elements of the parent movie.
Targeting Elements of Embedded Movies with All Wired Actions
All wired actions may be performed on elements of embedded movies, and wired action handlers inside of embedded movies may perform actions on elements of their parent movie.
To understand this targeting hierarchy, we can look at the current runtime state of a movie as a tree, as shown in the example in Figure 7-2. The root of this tree is the root movie itself. The children of the root movie node are tracks. Some types of tracks, such as the sprite track, can contain child nodes that are sprites. In earlier versions of QuickTime, this was as deep as the tree ever got, but with the introduction of the movie track, the tree can be indefinitely deep. The movie track node has child track nodes based on its currently loaded movie; one or more of these tracks may be yet another movie track.
Take, for example, a movie containing a Flash track and a movie track. The movie track’s currently loaded movie contains a video track, an audio track, and a sprite track with two sprites. This constitutes the target hierarchy tree shown in Figure 7-2.
Conceptually, you may think of the movie track and its currently loaded movie as a single entity––that is, a movie track. A movie track such as the child movie in this example may be the target of both track and movie actions. When specifying a target for an action, you first specify the movie (or movie track) that is the target or that contains the target. Then, you can further specify a track and track object if needed.
These are the ways you can target various elements within a hierarchy:
external movies by movie name or movie ID
child movie track by track ID, track index or track name
movies higher in the hierarchy as the parent movie or the root movie
These targets are relative to the current movie that contains the action handler. A few examples may help clarify how to target various elements.
Example 1: Sprite 1 has an action handler on a mouse click that tells the root movie to play.
Since Sprite 1 is contained in the child movie, the movie target is specified relative to the child movie. This may be accomplished by targeting the parent movie or root movie.
Example 2: Sprite 2 has an action handler that tells the Flash track to pan left.
Again, the target is relative to the child movie, so you can use either the parent movie or the root movie to specify the root movie. Additionally, you specify the Flash track.
Example 3: A button in the Flash track contains a mouse click handler that sets the volume of the audio track in the child movie. In this example, Flash track is contained in the root movie, so the movie target is specified relative to the root movie. You can use any of the child movie target types to specify the child movie. You can specify that the target is the audio track.
Target Type Atoms for Hierarchical Movies
There are target atoms to accommodate embedded movies in QuickTime. They allow for paths to be specified in a hierarchical movie tree.
Target movies may be an external movie, the default movie, or any movie embedded within another movie. Targets are specified using a movie path that may include parent and child movie relationships, and may additionally include track and track object target atoms as needed.
By using embedded kActionTarget
atoms along with parent and child movie target atoms, you can build up paths for movie targets. Note that you look for these embedded kActionTargetAtoms
only when evaluating a movie target, and any movie target type may contain a sibling kActionTargetAtom
.
Paths start from the current movie, which is the movie containing the object that is handling an event. You may go up the tree using a kTargetParentMovie
atom or down the tree using one of five child movie atoms. You may use a kTargetRootMovie
atom as a shortcut to get to the top of the tree containing an embedded movie and may use the existing movieByName
and movieByID
atoms to specify a root external movie.
The target atoms are described below. Note that there are five ways to specify an embedded child movie. Three of them specify movie track properties. Two specify properties of the currently loaded movie in a movie track.
- kTargetRootMovie (leaf atom –– no data)
The root movie containing the action handler.
- kTargetParentMovie (leaf atom –– no data)
The parent movie.
kTargetChildMovieTrackName
[Pstring movieTrackName]
A child movie track specified by track name.
kTargetChildMovieTrackID
[QTAtomID movieTrackID]
A child movie track specified by track ID.
kTargetChildMovieTrackIndex
[long movieTrackIndex]
A child movie track specified by track index.
kTargetChildMovieMovieName
[Pstring movieName]
A child movie specified by the currently loaded movie’s movie name. The child movie must contain
movieName
user data with the specified name.kTargetChildMovieMovieID
[QTAtomID movieID]
A child movie specified by the currently loaded movie’s movie ID. The child movie must contain
movieID
user data with the specified ID.
Example #1
Movie "Root" contains two embedded movies. "Controller" is an embedded movie controller movie. "ControlMe" is an embedded audio/video movie to be controlled. "Controller" and "ControlMe" are both child movies of the Root Movie.
To control the rate of the movie’s audio/video, a sprite in the “Controller” movie uses the following target:
kActionTarget |
kTargetParentMovie |
kActionTarget |
kTargetChildMovieMovieName |
[movieName = "ControlMe"] |
Example #2
A sprite in one movie targets a sprite which is embedded in a movie which is again embedded in another movie.
You target a sprite named "Dude" in a track of index 1 in a movie of ID 2 which is nested in a movie whose track name is "Rad", which is nested in a movie whose user data name is OuterExternalMovie
. OuterExternalMovie
is an external movie to the one that is executing an action handler.
kActionTarget |
kTargetMovieName |
[name = "OuterExternalMovie"] |
kActionTarget |
kTargetChildMovieTrackName |
[name = "Rad"] |
kActionTarget |
kTargetChildMovieMovieID |
[ID = 2] |
kTargetTrackIndex |
[Index = 1] |
kTargetSpriteName |
[spriteName = "Dude"] |
Movie Track and Movie Wired Actions
Two actions operate on a movie track target:
kActionMovieTrackAddChildMovie (QTAtomID childMovieID, CStr childMovieURL)
This action adds a movie URL data reference to the targeted movie track’s array of movie data references. The URL data reference is added with the specified ID. If a data reference with the same ID already exists, it is replaced. It is generally a good idea to use an ID that is not contained in the movie track's sample, since this may be reloaded under some conditions.
kActionMovieTrackLoadChildMovie (QTAtomID childMovieID)
This action loads a movie specified by childMovieID
as the current movie being played by the movie track. The movie replaces the current movie.
Another action operates on a root movie or a movie track target:
kActionMovieRestartAtTime (TimeValue startTime, Fixed rate) |
This action restarts the targeted movie at the specified movie time, restarting it at the specified rate. If rate
is set to 0, then the current movie rate is used. More specifically, this action stops the current movie, changes the movie’s time to the specified time, and then prerolls the movie from that time at the specified rate.
Note that the wired actions DoScript
, GotoURL
, DebugString
, and StatusString
are sent through the root movie’s MCDoActionProc
when executed from a child movie, allowing them to work with existing applications that trap for these actions.
Movie Controller Actions
mcActionDoScript
allows a movie to send requests to execute scripts of various sorts to a host application. The parameter is of type QTDoScriptPtr
.
struct QTDoScriptRecord { |
long scriptTypeFlags; |
char *command; |
char *arguments; |
}; |
typedef QTDoScriptRecord *QTDoScriptPtr; |
These are the constants currently defined for the scriptTypeFlags
field:
enum { |
kScriptIsUnknownType = 1L << 0 |
kScriptIsJavaScript = 1L << 1, |
kScriptIsLingoEvent = 1L << 2, |
kScriptIsVBEvent = 1L << 3, |
kScriptIsProjectorCommand = 1L << 4 |
}; |
For more information, see the explanation above of the new wired action kActionDoScript
.
mcActionRestartAtTime |
This allows a movie to be restarted at a particular time and rate.
The parameter is a QTRestartAtTimePtr.
struct QTRestartAtTimeRecord { |
TimeValue startTime; /* time scale is the movie timescale*/ |
Fixed rate; /* if rate is 0, the movie's current |
rate is maintained*/ |
}; |
typedef struct QTRestartAtTimeRecord QTRestartAtTimeRecord; |
typedef QTRestartAtTimeRecord *QTRestartAtTimePtr; |
For more information, see the explanation above of the new wired action kActionMovieRestartAtTime
.
Wired QT Event
kQTEventMovieLoaded
event was added to QuickTime 4.1. This event is sent when an embedded movie is loaded. Embedded movies may be loaded when a movie containing an embedded movie is first opened, when an embedded movie loads a new sample due to the movie’s time changing, or when an embedded movie is sent a kActionMovieTrackLoadChildMovie
action.
Action handlers for the kQTEventMovieLoaded
event may reside in two places. They may be placed in a sample of a movie track’s media or placed in the movie property atom of a movie that is loaded into a movie track.
If handlers exist in both places, the action list from the sample is appended to the one from the movie property atom. This means that the actions from the sample will be executed second, and if the same action resides in both places for the same target, the effect of the action from the sample will persist.
To add an event handler to a movie media sample, you add an atom of type kQTEventMovieLoaded
to the sample, with child atoms defining the actions.
To add an event handler to a child movie that is to be loaded, you add an atom of type kQTEventMovieLoaded
to the child movie’s movie property atom using the new QuickTime Movie Toolbox routine SetMoviePropertyAtom
. You may use the GetMoviePropertyAtom
function to first retrieve the existing movie properties container, add or modify the kQTEventMovieLoaded
atom, and then write it back using the SetMoviePropertyAtom
function.
Extended Wired Operand Functionality
A special case was added to the wired operand kOperandComponentVersion
.
By using the arguments kOperandComponentVersion("mac ", "os ", "vers" )
the version of Mac OS is returned. 0 is returned on Windows.
Wired Actions and JavaScript
kActionDoScript (long flags, CStr commands, CStr arguments) |
This new wired action has no target (system target).
The action calls the root movie controller’s mcActionDoScript
, so that scripts can be invoked by a host application. For example, the QuickTime Plug-in in a browser can invoke JavaScripts.
If the script flags are set to kScriptIsUnknownType
or kScriptIsJavaScript
, the QuickTime Plug-in invokes a JavaScript routine in the HTML file that has embedded the QuickTime movie, with the following prototype:
function DoFSCommand(command, arguments) { } |
If the movie is embedded with a NAME tag, a movieName
tag, or is named by user data, then this prototype is used instead:
function movieName_DoFSCommand(command, arguments) { } |
This allows for wired movies to invoke a JavaScript. The QuickTime Plug-in also supports many movie-related JavaScript routines, so it is possible to set sprite track variables and post custom wired events to a wired movie from JavaScript as well. It is important to note that this functionality works only with the QuickTime Plug-in and that some versions of some browsers do not support the necessary interfaces to allow for these things to work.
Movie Property Atom Toolbox Routines
Similar to the GetMediaPropertyAtom
and SetMediaPropertyAtom
routines, QuickTime 4.1 introduced GetMoviePropertyAtom
and SetMoviePropertyAtom
routines. These routines allow an atom container of structured data to be associated with a movie. This information is saved in a movie resource.
The kQTEventMovieLoaded
looks for an atom of type kQTEventMovieLoaded
in a movie's property atom container when a MovieTrack loads a new movie.
GetMoviePropertyAtom (Movie theMovie, QTAtomContainer * propertyAtom) |
This routine allocates and returns in propertyAtom
an atom container containing a copy of theMovie
’s properties’ atom container.
SetMoviePropertyAtom (Movie theMovie, QTAtomContainer propertyAtom) |
This routine sets the contents of theMovie
’s property atom container to the contents of the propertyAtom
atom container.
Custom Wired Actions
Developers may supplement the set of built-in wired actions by using a plug-in mechanism. You may write custom action handler components to perform new types of actions. For example, you could write a math library component to perform complicated computations quickly, returning the result by setting a sprite track variable. Another use would be for allowing a custom media handler to be scripted using wired actions.
Custom wired actions in a movie are routed to these components for handling. They are passed information about the current QTEvent
, the movie element that is to be the target of the action, the default movie element that received the QTEvent
and generated the action, the type of the action, and the parameters of the action. The wired action expression evaluation machinery has been exposed as a single API call, allowing general wired expressions to be passed as parameters.
Custom Action Handler Usage
Custom action handler usage falls into one of two categories:
as a stateless subroutine library
as an object that maintains state across multiple custom action executions
When used as a subroutine library, the movie author simply scripts custom actions. QuickTime opens an instance of the specified custom action handler, passes it the action to execute, and then closes the component. When used as an object that maintains state, the movie author needs to open an instance of the handler with a unique instance ID. In this case, QuickTime keeps the action handler component open until the movie is closed. When making calls to an action handler that has been opened, the unique instance ID is specified, allowing QuickTime to route the request to the correct component instance.
Note that there is currently no way to specify that the handler is a component that has already been opened by QuickTime, such as a media handler or codec that is in use. One special case has been included for developers authoring media handlers that support custom wired actions. If the component description of the custom action’s target matches that of the target media handler, then the custom action is sent to the instance of the media handler that already in use. This means that the movie author does not need to (and should not) open an instance of the component.
When authoring a movie with a custom action, the type of component used to execute it is specified, along with an instance ID. If the component is being used as a simple subroutine library, which does not need to keep track of any state, then the ID may be set to 0. If a particular instance of an open action-handling component is intended to be used, then the ID is used to refer to it. You may use the kActionOpenCustomHandler
to open the component with a particular ID. This ID should be obtained by using the kOperandUniqueCustomHandlerID
to ensure that your movie will work after edits are performed. After opening a custom action handler you may use kOperandCustomActionHandlerOpen
to determine that the component was indeed found and opened successfully.
If the instance ID is set to 0 and the component description matches that of the media handler that is the target of the action, then the media handler is used to execute the custom action.
Authoring Custom Wired Actions
To specify that an action is to be handled by a custom handler, you add the kCustomActionHandler
atom as a child of the kAction
atom. You define which type of component is to handle the action by adding a kCustomHandlerDesc
atom, and optionally specify a particular handler instance ID using the kCustomHandlerID
atom.
Extension to Wired Movie Format: Executing Custom Actions
kActionAtom |
<kCustomActionHandler, 1, 1> |
kCustomHandlerDesc, 1, 1 |
[ComponentDescription handlerDesc] |
<kCustomHandlerID, 1, 1> |
[long handlerID] |
You define parameter atoms as usual and have access to the parameters through an API for writing custom action handlers. This means that parameters may be wired expressions and your component may fetch the result of the evaluated expression.
Wired Actions
kActionOpenCustomHandler |
Supported Flags: None |
Param 1: [long handlerID] |
Param 2: [QTCustomActionHandlerRecord |
Opens a custom action-handling component that may be referred to later by its handler ID.
Typically, you would first use the kOperandUniqueCustomActionHandlerID
operand to obtain a unique ID. After storing this in a variable, you use this action to open the handler. Then you can use kOperandCustomActionHandlerOpen
to determine if the handler was found.
Wired Operands
kOperandUniqueCustomActionHandlerID |
No Params |
Returns a unique custom handler ID that may be used with kActionOpenCustomHandler
.
kOperandCustomActionHandlerOpen |
Param 1: handlerID |
Returns true
if a handler with the specified ID is open, otherwise false
. This may be used to determine if the component specified by kActionOpenCustomHandler
was found and opened.
Writing a Custom Action Handler Component
Any component type, whether it is a media handler, a codec, or your own component type, may be extended to handle custom actions. To extend your component, you implement the ExecuteWiredAction
routine described below. If you are writing a component that is being used only to handle custom actions, you should use the component type 'wire'
.
EXTERN_API( ComponentResult ) |
CallComponentExecuteWiredAction (ComponentInstance ci, |
QTAtomContainer actionContainer, |
QTAtom actionAtom, |
QTCustomActionTargetPtr target, |
QTEventRecordPtr event); |
All the state passed to you in this routine is only valid for the duration of the execution of your custom action, which should be completed when you return from this routine.
The actionContainer
contains all of the actions that are being executed in response to the current QTEvent
. This atom container should not be edited, only read from, since other actions that have yet to be executed may be contained within it.
The actionAtom
specifies the kActionAtom
that concerns you, since it contains all of the atoms describing the action type and parameters to your components custom action that are to be executed.
The target
parameter specifies the movie elements that you need in order to execute your action.
struct QTCustomActionTargetRecord { |
Movie movie; |
DoMCActionUPP doMCActionCallbackProc; |
long callBackRefcon; |
Track track; |
long trackObjectRefCon; |
Track defaultTrack; |
long defaultObjectRefCon; |
long reserved1; |
long reserved2; |
}; |
typedef struct QTCustomActionTargetRecord QTCustomActionTargetRecord; |
typedef QTCustomActionTargetRecord * QTCustomActionTargetPtr; |
The movie
field is the movie that is or contains the movie element, which is the target of the action.
The doMCActionCallbackProc
and callBackRefcon
specify the movie controller and refCon
for the target movie’s movie controller. They may be used with the CallDoMCActionProc
routine to invoke the movie controller functionality, including the new mcActionFetchParameterAs
.
The track
field is the track that is or contains the movie element, which is the target of the action.
The trackObjectRefCon
field is the refCon
or ID of the movie element that is the target of the action. If the target is a sprite, this will be the sprite ID.
The defaultTrack
is the track that contains the movie element that handled the QTEvent
and generated the custom action. For example, if a hypertext element of a text track generated a SetSpriteVisible
action for a sprite in a sprite track, the defaultTrack
would be the text track, while the sprite track would be the target track.
The defaultObjectRefCon
is the refCon
or ID of the movie element that handled the QTEvent and generated the custom action.
The event
specifies information about what QTEvent
generated the custom action.
The Action Being Executed
If your component defines multiple custom action types, then you can determine the action type by looking at the kWhichAction
atom, which is a child atom of this kActionAtom
. The data of this kWhichAction
atom is a long
defining the action (and needs byte flipping for Windows).
Fetching the Parameters
The parameters to your custom action may be the result of a wired expression. You don’t have to be concerned about analyzing the kActionParameter
child atoms: the wired expression evaluation machinery has been made accessible via a movie controller action called mcActionFetchParameterAs
.
You fill out a QTFetchParameterAsRecord
and pass it to mcActionFetchParameterAs
as in:
CallDoMCActionProc(resolvedTarget->doMCActionCallbackProc, |
resolvedTarget->callBackRefcon, |
mcActionFetchParameterAs, |
&fetchAs, &handled ); |
struct QTFetchParameterAsRecord { |
QTAtomSpec paramListSpec; |
long paramIndex; |
long paramType; |
long allowedFlags; |
void * min; |
void * max; |
void * currentValue; |
void * newValue; |
Boolean isUnsignedValue; |
}; |
You set the container and atom of the paramListSpec
to the actionContainer
and actionAtom
passed to ExecuteWiredAction()
.
You set the paramIndex
to the index of the parameter you wish to fetch. If you allow a variable number of parameters, you may count how many child atoms of type kActionParameter
the actionAtom
has.
You set the paramType
to one of the parameter types from the following enumeration:
enum { |
kFetchAsBooleanPtr = 1, |
kFetchAsShortPtr = 2, |
kFetchAsLongPtr = 3, |
kFetchAsMatrixRecordPtr = 4, |
kFetchAsModifierTrackGraphicsModeRecord = 5, |
kFetchAsHandle = 6, |
kFetchAsStr255 = 7, |
kFetchAsFloatPtr = 8, |
kFetchAsPointPtr = 9, |
kFetchAsNewAtomContainer = 10, |
kFetchAsQTEventRecordPtr = 11, |
kFetchAsFixedPtr = 12, |
kFetchAsSetControllerValuePtr = 13, |
kFetchAsRgnHandle = 14, /* flipped to native*/ |
kFetchAsComponentDescriptionPtr = 15, |
kFetchAsCString = 16 |
}; |
You set the allowedFlags
to flags from the following enumeration, or 0 fetching without constraints.
enum { |
kActionFlagActionIsDelta = 1L << 1, |
kActionFlagParameterWrapsAround = 1L << 2, |
kActionFlagActionIsToggle = 1L << 3 |
}; |
If allowedFlags
is not set to 0, you set the min
, max
, current
, and isUnsignedValue
fields to appropriate values based on the data type being fetched.
The newValue
field returns the result of the parameter.
For scalar and structure types, you pass a pointer to the appropriate data type and it will be filled in.
For Handle
, RgnHandle
, and Cstring
, you pass in a new empty handle that will be resized as needed, you are responsible for disposing the handle when done.
For kFetchAsNewAtomContainer
, you pass in a pointer to a non-allocated QTAtomContainer
. On return, this container will contain the contents of the single child atom of the parameter atom and all of its children. This lets you pass arbitrary data that is not evaluated in an atom container as a parameter.
Copyright © 2003, 2009 Apple Inc. All Rights Reserved. Terms of Use | Privacy Policy | Updated: 2009-06-01