Cubic QuickTime VR Movies

This chapter is aimed at QuickTime VR developers and programmers who want to create cubic QuickTime VR movies. It discusses some of the key features of the MakeCubic utility application provided by Apple, which enables you to produce cubic QTVR movies from cube faces and from equirectangular spherical pictures. The chapter also explains some of the techniques you can use to convert a panoramic image into a QuickTime VR panoramic movie. To accomplish this, Apple provides free sample code that you can download at

http://developer.apple.com/samplecode/Sample_Code/QuickTime/QuickTime_VR.htm

The sample code includes the VRMakePano.h and the VRMakePano.c files, and is part of the VRMakePano library code.

As discussed briefly in the section QuickTime VR and in more detail in Cubic Panoramas, QuickTime 5.01 introduced a new cubic playback engine which stores panoramic images as six or more separate images that can be projected during playback onto the sides of a cube (polyhedron), thus enabling the user to look straight up and straight down––with spectacular results. Imagine looking skyward straight up to the top of the Eiffel Tower or seeing the full dimensions, floor and ceiling, of any room or cubic space.

The QuickTime VR cubic playback engine was designed to be backward compatible, so that properly constructed cubic VRs would still be able to play in earlier versions of QuickTime as panoramas––with slight distortion.

This chapter is divided into the following major sections:

Overview

To assist developers who are working with QuickTime VR, Apple provides the following:

The process of making a cylindrical or cubic panorama generally involves these steps:

  1. You create the movie file for the destination.

  2. Create the QTVR movie track and the media for it.

  3. Create the panorama track and the media for it.

  4. Add the media for the video.

  5. Add the controller.

  6. Flatten it.

The VRMakePano.c and VRMakePano.h sample code provides a common interface, with consistent procedure names, for example

OSErr   VRMovieToQTVRCubicPano(
    VRMakeQTVRParams    *qtvrParams,
    FSSpec              *srcTileSpec,
    FSSpec              *srcHSTileSpec,
    FSSpec              *srcFSTileSpec,
    FSSpec              *dstMovieSpec
);

The procedure names in the VRMakePano.h API are consistently formed, as shown in Figure 6-1.

Figure 6-1  The procedure names in the VRMakePano.h API
The procedure names in the VRMakePano.h API

The common parameters (VRMakeQTVRParams) are

{pan,tilt,FOV} {min,max,default}
tiles           {numH, numV, codec, quality}
preview         {codec, quality}
name            {scene, node}
quality         {dynamic, static}
window          {width,height}
flattener       {flags, previewResolution}
hotSpots
trackDuration

These parameters apply to all types of panoramas (cylindrical and cubic), as well as sources (movies, PICT files, and GWorlds).

MakeCubic Utility Application

MakeCubic is a simple utility application that is provided by Apple for developers who want to create cubic QuickTime VR panoramas from six faces or from equirectangular (a kind of sphere-to-rectangle projection) images. It is available for download from the Apple developer website at

http://developer.apple.com/quicktime/quicktimeintro/tools/index.html

MakeCubic performs tiling, compression, and preview creation for cubic VR. To use the default settings, you drag an equirectangular image, or six cubic face images onto the application.

To adjust the tiling, compression, or preview parameters, you launch the application by double-clicking its icon and selecting Convert from the File menu. The MakeCubic parameters dialog is shown in Figure 6-2.

The MakeCubic application provides some features beyond those found in the VRMakePano.c Library, including

Some VRMakePano functions that are not accessible in the MakeCubic application:

Figure 6-2  The MakeCubic Panorama Parameters dialog
The MakeCubic Panorama Parameters dialog

MakeCubic is designed primarily for QuickTime VR developers who are using another stitcher program that generates an equirectangular image and for those who are synthesizing and generating the cube faces directly, as with a 3D renderer.

The MakeCubic application has a number of settings in the Panorama Parameters dialog, which are briefly explained as follows:

Generally, all image files will be of the same type––that is, JPEGs. You would not normally mix image files––for example, JPEGs, GIFs, and TIFFs––because you would end up with unacceptable compression artifacts, though MakeCubic can deal with it.

The MakeCubic Preferences dialog, shown in Figure 6-3, has settings that are explained as follows:

Figure 6-3  MakeCubic Preferences dialog
MakeCubic Preferences dialog

Using the VRMakePano.c Library

VRMakePano.c is code provided by Apple that is designed primarily to serve as a library for developers who want to convert cylindrical panoramic images into QuickTime VR panoramic movies. The code compiles and runs on both the Mac OS and Windows platforms and has minimal dependencies and a rich API.

Using the code supplied in VRMakePano.c, you can also construct a QuickTime VR movie from the six faces of a cube, so that the movie can be viewed using the cubic projection engine introduced in QuickTime. There are three different types of sources that can be supplied to VRMakePano.c: GWorlds, image files, and movies.

With each of these interfaces, you can supply

For movie files, you have one movie file for each of the three tracks; with picture files, you may have one file (equirectangular), or more than one file (faces); with GWorlds, you assume that you always have six GWorlds for the cubic, or one GWorld for the cylindrical, just the same as for picture files.

The interface is designed to be orthongonal, so that it looks the same regardless if there are PICT files, or GWorlds, whether they are cubic or cyclindrical. Thus, the interface is the same, regardless of the files you have.

Most parameters are encapsulated in a standard parameter data structure:

    VRMakeQTVRParams

All fields should be set. For example, the following settings would be suitable for cubics:

    p.tilesH             = 1;
    p.tilesV             = 1;
    p.tileCodec          = kJPEGCodecType;
    p.tileQuality        = codecNormalQuality;
    p.sceneName          = NULL;
    p.nodeName           = NULL;
    p.dynamicQuality     = codecNormalQuality;
    p.staticQuality      = codecHighQuality;
    p.trackDuration      = 7200;
    p.previewCodec      = JPEG
    p.previewQuality    = low
    p.wraps              = 1;
    p.minPan             = 0.0f;
    p.maxPan             = 360.0f;
    p.minTilt            = -90.0f;
    p.maxTilt            =  90.0f;
    p.minFieldOfView     =   5.0f;
    p.maxFieldOfView     = 120.0f;
    p.defaultPan         =   0.0f;
    p.defaultTilt        =   0.0f;
    p.defaultFieldOfView =  60.0f;
    p.windowWidth        = 480;
    p.windowHeight       = 320;
    p.hotSpots           = NULL;
    p.flattenerFlags     = kVRMakePano_GeneratePreview |
                            kVRMakePano_BlurGeneratedPreview;
    p.flattenerPreviewResolution = 4;

Typical usage would then be:

    err = VRXXXToQTVRYYYPano(&p, &srcSpec, &srcHSSpec, &srcFSSpec,
                                &dstSpec);

The API does not currently include the ability to set the hot spot codec (default: graphics 100%).

Cubic Panorama File Format

As discussed, QuickTime enables you to use cubic panoramas, which store the panoramic image as six or more separate images that are projected during playback onto the sides of a cube (polyhedron), allowing the user to look straight up and straight down. The file format for cubic panoramas is identical with the Version 2.0 cylindrical file format, with these exceptions:

  • The pano track identifies the type of pano as 'cube'.

  • The pano track has a 'cuvw' atom to specify the actual view parameters, allowing the pano data sample’s view parameters to be assigned backward-compatibility values for QuickTime 4.x, which can only render cylindrical panoramas.

  • The pano track optionally has a 'cufa' atom to specify the placement of the faces. This may be omitted if there are six faces and they appear in the standard order.

For cubic panoramas, some of the fields in the panorama sample atom should be assigned special values that allow the file to be displayed with the cylindrical engine if the cubic engine is not available. The cubic engine ignores those values, instead using values stored in the new cubic view atom.

Inside VRMakePano.c

VRMakePano.ccontains code for creating a QuickTime VR panoramic movie from a panoramic image. The image can be a picture of type 'PICT' or any other kind of image for which QuickTime has a graphics importer component. You can also create hot spot image tracks and assemble the various hot spot atoms. This file also contains a function that constructs a QuickTime VR movie that uses the cubic projection available in QuickTime––that is, cubic movies or cubic panoramas.

A panoramic movie contains at least three tracks: a QTVR track, a panorama track, and a panorama image track. In addition, a QuickTime VR movie must contain some special user data that specifies the QuickTime VR movie controller. A QuickTime VR movie can also contain other kinds of tracks, such as hot spot image tracks, preview tracks, and even sound tracks.

A QuickTime VR movie contains a single QTVR track, which maintains a list of the nodes in the movie.

Each individual sample in the QTVR track’s media contains information about a single node, such as the node’s type, ID, and name. Since you are creating a single-node movie here, your QTVR track will contain a single media sample.

Every media sample in a QTVR track has the same sample description, whose type is QTVRSampleDescription. The data field of that sample description is a VR world, which is an atom container whose child atoms specify information about the nodes in the movie, such as the default node ID and the default imaging properties.

A panoramic movie also contains a single panorama track, which contains information specific to the panorama. A panorama track has a media sample for each media sample in the QTVR track. As a result, our panorama track will have one sample. The QTVRPanoSampleAtom structure defines the media sample data.

The actual image data for a panoramic node is contained in a panorama image (video) track. The individual frames in that track are the diced (and compressed) tiles of the original panoramic image.

There may also be a hot spot image track that contains the diced (and compressed) tiles of the hot spot panoramic image.

The general strategy, given a panoramic image, is as follows:

  1. Create a movie containing a video track whose frames are the compressed tiles of the panoramic image. Call this movie the “tile movie.” Create a similar movie for the hot spot image. Call this movie the “hot spot tile movie.” Similarly, for the preview.

  2. Create a new, empty movie. Call this movie the “QTVR movie.”

  3. Create a QTVR track and its associated media.

  4. Create a VR world atom container; this is stored in the sample description for the QTVR track.

  5. Create a node information atom container for each node; this is stored as a media sample in the QTVR track.

  6. Create a panorama track and add it to the movie.

  7. Create a preview track.

  8. Create a panorama image track by copying the video track from the tile movie to the QTVR movie.

  9. Create a hot spot image track by copying the video track from the hot spot tile movie to the QTVR movie.

  10. Set up track references from the QTVR track to the panorama track, and from the panorama track to the panorama image track and the hot spot image track and preview.

  11. Add a user data item that identifies the QTVR movie controller.

  12. Flatten the QTVR movie into the final panoramic movie.

Specifying the Faces of a Cube

The code in Listing 6-1 deals with the vectors and quaternions that are used to specify the faces of a cube. The interface describes an axis and an orientation around that axis in order to set where the face is.

For example, this particular interface would be used to specify right, left, back, and front in a mathematically defined way. To do the main four faces, you would set the x, y, z of the axis of rotation to be [0, 1, 0], namely, pointing up on the y axis. So the front face would be rotated by 0 around that axis, the next ones would be rotated by 90 degrees from the previous. From this interface, it can generate all the data needed to set the face orientation, or sub-face.

This example assumes that the faces are equal subtiles of six cube faces.

Listing 6-1  Specifying the faces of a cube

SetOneCubicFaceData(
    QTVRCubicFaceData   *face,
    float               x,   /* axis of rotation */
    float               y,
    float               z,
    float               degrees, /* rotation about axis */
    long                tilesH, /* subdivisions per face */
    long                tilesV,
    long                h,  /* The horizontal index of this sub-face */
    long                v   /* The vertical   index of this sub-face */
)
{
    double halfAngle, norm, sqrtCotVFOV, s, c;
 
    sqrtCotVFOV = sqrt((double)tilesV);
    halfAngle = degrees * pi / 360.0;
    norm = x*x + y*y + z*z;
    if (norm != 0.0)
        norm = 1.0 / sqrt(norm);
    if (fabs(s = sin(halfAngle)) < 1.0e-8) s = 0.0; /* make nice */
    if (fabs(c = cos(halfAngle)) < 1.0e-8) c = 0.0; /* file values*/
    norm *= s * sqrtCotVFOV;
 
    face->orientation[0] = EndianF32_NtoB(c * sqrtCotVFOV);
    face->orientation[1] = EndianF32_NtoB(x * norm);
    face->orientation[2] = EndianF32_NtoB(y * norm);
    face->orientation[3] = EndianF32_NtoB(z * norm);
 
    /* Center, normalized by the vertical dimension */
    face->center[0] = EndianF32_NtoB( (2 * h - tilesH + 1) * (float)tilesV
                                        / (float)tilesH);
    face->aspect = EndianF32_NtoB(1.0f);
    face->skew = EndianF32_NtoB(0.0f);
}

Converting a Tile Movie To a Cylindrical QuickTime VR Movie

The code in Listing 6-2 converts a tile movie to a cylindrical QuickTime VR movie. You can also specify an optional hot spot tile movie and/or fast start movie. The window dimensions, tiling, track duration, compression codec, compression qualities and rendering qualities (static and dynamic) are mandatory.

The tiles are assumed not to be rotated. (The “2vo” version should be used if the tiles are rotated.) For a wrapping panoramic image of VFOV < 145 degrees, this means that the horizontal dimension (circumference) is larger than the vertical (axis).

Using the VRMovieToQTVRCylPano2h0 or VRMovieToQTVRCylPano2v0functions, you can create a single-node panoramic QTVR movie from the specified tile movies. The VRMovieToQTVRCylPano2h0 function builds a movie that conforms to version 2.0 of the QuickTime VR file format. VRMakePano.c also contains code to make version 1.0 files, but this is discouraged.

The newly-created movie contains references to the original tile movie––not the movie data itself. This is done because the assumption is that the caller will flatten the movie into a third movie, which will contain the movie data. Also, the interim file is much smaller than it would be if the data is copied, thus saving time and disk space.

Listing 6-2  Code to convert a cylindrical movie to a cylindrical panorama movie, with rotated source

VRMovieToQTVRCylPano2h0(
    VRMakeQTVRParams *qtvrParams, /* Parameters to create the movie */
    FSSpec  *srcTileSpec,   /* Cylindrical panoramic tile movie */
    FSSpec  *srcHSTileSpec, /* Cylindrical hot spot tile movie */
    FSSpec *srcFSTileSpec,  /* Cylindrical fast start tile movie */
    FSSpec  *dstMovieSpec   /* Destination movie */
)
{
    short               myResRefNum = -1;
    Movie               tmpMovie = NULL;
    Track               myQTVRTrack;
    Track               myPanoTrack;
    ComponentResult     err  = noErr;
    FSSpec              tmpSpec;

These are the steps you follow:

  1. You create a movie file for the destination movie.

        MakeTempFSSpec(dstMovieSpec, ".MV~", &tmpSpec);
        err = CreateMovieFile(&tmpSpec, FOUR_CHAR_CODE('TVOD'),
                                smCurrentScript, kCreateMovieFlags,
                                &myResRefNum, &tmpMovie);
  2. Create the QTVR movie track and media.

        err = CreateQTVRTrack(qtvrParams, qtvrParams->trackDuration,
                                tmpMovie, &myQTVRTrack);
  3. Create the panorama track and the media, and add them to the movie.

        err = CreatePanoTrackFromMovies(srcTileSpec, srcHSTileSpec,
                                        srcFSTileSpec,
                                        qtvrParams, panoType, tmpMovie,
                                        myQTVRTrack, &myPanoTrack);
  4. Add a user data item that identifies the QTVR movie controller.

        err = SetQTControllerType(tmpMovie, kQTVRQTVRType);
  5. Create the final, flattened movie from the temporary file into a new movie file; put the movie resource first, so that FastStart is possible.

        err = FlattenQTVRMovie(tmpMovie, qtvrParams->flattenerFlags,
                                qtvrParams->flattenerPreviewResolution,
                                dstMovieSpec);
     
    bail:
        if (myResRefNum != -1)      CloseMovieFile(myResRefNum);
        if (tmpMovie    != NULL)    DisposeMovie(tmpMovie);
        DeleteMovieFile(&tmpSpec);
     
        return(err);
    }

Converting Movies to Cubic Panorama Movies

The code in Listing 6-3 converts a movie with six frames into a cubic QuickTime VR panorama movie. An optional hot spot movie and/or fast start movie can also be specified. The window dimensions, tiling, track duration, compression codec, compression qualities and rendering qualities (static and dynamic) are mandatory.

Using the VRMovieToQTVRCubicPano function, you can create a single-node cubic panoramic QTVR movie from the specified six-frame movie.

Listing 6-3  Converting a movie with six frames into a cubic QuickTime VR panorama movie

VRMovieToQTVRCubicPano(
    VRMakeQTVRParams    *qtvrParams,/* Parameters to create the movie */
    FSSpec  *srcFramesSpec, /* Source movie with the panorama faces */
    FSSpec  *srcHSFramesSpec, /* Source movie with the hot spot faces */
    FSSpec  *srcFSFramesSpec,/* Source movie with the fast start faces */
    FSSpec  *dstMovieSpec   /* Destination movie */
)
{
    FSSpec  tmpSpec;
    short   myResRefNum = -1;
    Movie   tmpMovie = NULL;
    Track   tmpQTVRTrack;
    Track   tmpPanoTrack;
    ComponentResult  err;

These are the steps you follow:

  1. You create a temporary version of the panorama movie file, located in the same directory as the destination panorama movie file.

        MakeTempFSSpec(dstMovieSpec, ".MV~", &tmpSpec);
        err = CreateMovieFile(&tmpSpec, FOUR_CHAR_CODE('TVOD'),
                                smCurrentScript, kCreateMovieFlags,
                                &myResRefNum, &tmpMovie);
  2. Create the QTVR movie track and media.

        err = CreateQTVRTrack(qtvrParams, qtvrParams->trackDuration,
                                tmpMovie, &tmpQTVRTrack);
  3. Create panorama track and media, and add them to the movie.

    err = CreatePanoTrackFromMovies(srcFramesSpec, srcHSFramesSpec,
                                    srcFSFramesSpec, qtvrParams,
                                    kQTVRCubicVersion1, tmpMovie,
                                    tmpQTVRTrack, &tmpPanoTrack);
  4. Add a user data item that identifies the QTVR movie controller.

        err = SetQTControllerType(tmpMovie, kQTVRQTVRType);
  5. Create the final, flattened movie, from the temporary file into a new movie file; put the movie resource first, so that FastStart is possible.

        err = FlattenQTVRMovie(tmpMovie, qtvrParams->flattenerFlags,
                                qtvrParams->flattenerPreviewResolution,
                                dstMovieSpec);
    }

Converting Cubic Picture files to Cubic Panorama Movies

The code in Listing 6-4 converts a set of six picture files to a cubic QuickTime VR panorama movie.

An optional hot spot picture files and/or fast start picture files can also be specified. The window dimensions, tiling, track duration, compression codec, compression qualities and rendering qualities (static and dynamic) are mandatory.

There are certain restrictions on sizes, however, for tiling:

Adjacent tiles duplicate their edges; therefore, the GWorld size should be evenly divisible into tiles, taking this overlap into account. Tile size is:

t = (f + n - 1) / n

where

t is the tile size (width or height),

f is the face size (width or height is the same as above),

n is the number of tiles in that dimension (width or height).

For example, a 767x767 face is divided into 4=2x2 tiles of size 384x384, while a 766x766 face is divided into 9=3x3 tiles of size 256x256.

This implements tiling of the faces, but the face dimensions must be appropriately divisible:

    (width  - tilesH + 1) / tilesH = integer
    (height - tilesV + 1) / tilesV = integer

For example, {dim=512,tiles=1} , {dim=511,tiles=2} , {dim=510,tiles=3}.

Listing 6-4  Converting a set of six picture files to a cubic QuickTime VR panorama movie

VRPictsToQTVRCubicPano(
    VRMakeQTVRParams *qtvrParams,/* Parameters to create the movie */
    FSSpecHandle    srcPictSpecs,/* Source images */
    FSSpecHandle    srcHSPictSpecs, /* Hot spot images */
    FSSpecHandle    srcFSPictSpecs, /* Fast start images */
    FSSpec          *dstMovieSpec   /* Destination movie */
)
{
    FSSpec  tmpSpec;
    short   tmpRefNum   = -1;
    Movie   tmpMovie    = NULL;
    Track   qtvrTrack   = NULL;
    Track   panoTrack   = NULL;
    ComponentResult  err;

These are the steps you follow:

  1. You create a temporary version of the panorama movie file, located in the same directory as the destination panorama movie file.

        MakeTempFSSpec(dstMovieSpec, ".MV~", &tmpSpec);
        err = CreateMovieFile(&tmpSpec, FOUR_CHAR_CODE('TVOD'),
                                smCurrentScript, kCreateMovieFlags,
                                &tmpRefNum, &tmpMovie);
  2. Create the QTVR movie track and media.

        err = CreateQTVRTrack(qtvrParams, qtvrParams->trackDuration,
                                tmpMovie, &qtvrTrack);
  3. Create panorama track and media, and add them to the movie.

        err = CreatePanoTrackFromPicts(6, srcPictSpecs, srcHSPictSpecs,
                                        srcFSPictSpecs, kQTVRCubicVersion1,
                                        qtvrParams, tmpMovie, qtvrTrack,
                                        &panoTrack);
  4. Add a user data item that identifies the QTVR movie controller.

        err = SetQTControllerType(tmpMovie, kQTVRQTVRType);
  5. Create the final, flattened movie, from the temporary file into a new movie file; put the movie resource first, so that FastStart is possible.

    err = FlattenQTVRMovie(tmpMovie, qtvrParams->flattenerFlags,
                            qtvrParams->flattenerPreviewResolution,
                            dstMovieSpec);
    }

Converting GWorlds to Cubic Panorama Movies

The code in Listing 6-5 converts a set of six GWorlds to a cubic QuickTime VR panorama movie. An optional hot spot GWorlds and/or fast start GWorlds can also be specified.

The window dimensions, tiling, track duration, compression codec, compression qualities and rendering qualities (static and dynamic) are mandatory.

There are certain restrictions on sizes for tiling:

Adjacent tiles duplicate their edges; therefore, the GWorld size should be evenly divisible into tiles, taking this overlap into account. Tile size is:

 t = (f + n - 1) / n

where

t is the tile size (width or height),

f is the face size (width or height - same as above), and

n is the number of tiles in that dimension (width or height).

For example, a 767x767 face is divided into 4=2x2 tiles of size 384x384, while a 766x766 face is divided into 9=3x3 tiles of size 256x256.

Given six GWorlds (and six possible hot spot GWorlds), you can create a cubic panorama movie. This implements tiling of the faces, but the face dimensions must be appropriately divisible:

 (width  - tilesH + 1) / tilesH = integer
    (height - tilesV + 1) / tilesV = integer

For example, {dim=512,tiles=1} , {dim=511,tiles=2} , {dim=510,tiles=3}.

Listing 6-5  Code for converting a set of six GWorlds to a cubic QuickTime VR panorama movie

VRGWorldsToQTVRCubicPano(
    VRMakeQTVRParams *qtvrParams, /* Parameters to create the movie */
    GWorldPtr        *srcGWs,   /* 6 Source GWorlds in standard order */
    GWorldPtr       *srcHSGWs,  /* 6 Hot spot GWorlds in order */
    GWorldPtr       *srcFSGWs,  /* 6 Faststart GWorlds in order */
    FSSpec          *dstMovieSpec /* Destination movie */
)
{
    FSSpec              tmpSpec;
    short               tmpRefNum               = -1;
    Movie               tmpMovie                = NULL;
    Track               qtvrTrack               = NULL;
    Track               panoTrack               = NULL;
    ComponentResult     err;

These are the steps you follow:

  1. Create a temporary version of the panorama movie file, located in the same directory as the destination panorama movie file.

    MakeTempFSSpec(dstMovieSpec, ".MV~", &tmpSpec);
        err = CreateMovieFile(&tmpSpec, FOUR_CHAR_CODE('TVOD'),
                                smCurrentScript, kCreateMovieFlags,
                                &tmpRefNum, &tmpMovie);
  2. Create the QTVR movie track and media.

        err = CreateQTVRTrack(qtvrParams, qtvrParams->trackDuration,
                                tmpMovie, &qtvrTrack);
  3. Create panorama track and media, and add them to the movie.

        err = CreatePanoTrackFromGWorlds(6, srcGWs, srcHSGWs, srcFSGWs,
                                        kQTVRCubicVersion1,
                                        qtvrParams, tmpMovie, qtvrTrack,
                                        &panoTrack);
  4. Add a user data item that identifies the QTVR movie controller.

        err = SetQTControllerType(tmpMovie, kQTVRQTVRType);
  5. Create the final, flattened movie, from the temporary file into a new movie file; put the movie resource first so that FastStart is possible.

        err = FlattenQTVRMovie(tmpMovie, qtvrParams->flattenerFlags,
                                qtvrParams->flattenerPreviewResolution,
                                dstMovieSpec);
    }