Explore AirPlay with interstitials

♪ ♪ Amit: Hi, everyone. My name is Amit. I am an engineer on the AirPlay team. We created AirPlay to let you wirelessly share videos, photos, music, and more from Apple devices to your AirPlay-capable receivers. There are times at which you might want to play secondary content from multiple sources, like playing an ad, a trailer of an upcoming release, and more with the primary content in your application. This secondary content is called interstitials. If you are interested in ensuring your application has a great AirPlay user experience for interstitials, you have tuned in to the right session. In this talk, we'll start with a quick overview of AirPlay and HLS Interstitials. After that, we will discuss how navigation restrictions on interstitial playback can be forwarded to AirPlay receivers. We'll then go over the best practices to optimize AirPlay playback on all devices. We will briefly discuss about interstitial localization and AirPlay testing. Finally, we'll present an example of the VOD service plan offering, using HLS interstitials. You can use AirPlay in one of the following ways: AirPlay Audio. If you have more than one HomePod or AirPlay-enabled speaker, you can enjoy your favorite music or podcasts throughout your house in perfect sync. For more information on AirPlay Audio, you can check out the session "Tune up your AirPlay Audio experience" by my colleague Kelly. AirPlay mirroring allows you to share anything from your iOS or macOS screen such as websites, presentations, and spreadsheets to AirPlay receivers.

AirPlay Video helps you to wirelessly send a video to watch on your AirPlay capable TV, that is Apple TV or Mac receiver or AirPlay-enabled smart TV. This is a high-quality video experience with both Apple TV or AirPlay-capable TVs, which can support up to 4K HDR video with a surround sound. With HLS Interstitials, ads are treated as separate assets that are defined in the timeline of a primary media asset. When the timeline of the media player hits the marker, we'll begin playing the interstitials. HLS Interstitials can be scheduled against either VOD or live primary content with no modification required to primary content. Interstitials can be dynamically scheduled or unscheduled during playback. HLS Interstitials allows you for late binding of your ad assets, where you'd like for an ad to be decisioned as the primary content is approaching to the ad marker, and even rebinding. For example, when people navigate to a portion of video that they have already watched, using HLS interstitials, you can ensure that they're not going to see the same ad all over again. HLS interstitials offers built-in support for AirPlay and Picture in Picture. All of the scheduled ads and interstitials shall be carried over to Airplay session. HLS Interstitials can be used for audio and video streaming services.

HLS Interstitials can be scheduled in one of the two ways: server-driven, client-driven. Both the methods have similarity in terms of setting properties for scheduling and controlling the presentation of interstitial. Servers can schedule interstitials by placing EXT-X-DATERANGE tags into the media playlist of the primary asset. Application code can use AVPlayerInterstitialEventMonitor object to monitor the scheduling and the progress of interstitials. Application code can schedule interstitials programmatically by using AVFoundation APIs. You can create an ad event by creating an object of AVPlayerInterstitialEvent. AVPlayerInterstitialEvent object contains all the information necessary to place ads on a player's item timeline. You can set all the objects of AVPlayerInterstitialEvent on the AVPlayerInterstitialEventController. AVPlayerInterstitialEventController schedules interstitial events for the items played by the primary player.

Here is an example Playback flow for VOD and Live content. As you can you see, we have back-to-back ad assets, that is ad1 and ad2, for both VOD and Live use case. For VOD use case, playback is resumed at the point it left off. For the Live use case, playback jumps forward by the time consumed by ads. Resumption offset of primary content can be set by server-driven or client-driven method. For server method, the resumption offset for the primary content can be achieved by setting X-RESUME-OFFSET attribute within the EXT-DATERANGE tag in the HLS Playlist. For client method, the resumption offset can be set using AVFoundation API, that is by setting resumptionOffset property into the AVPlayerInterstitialEvent. For details of HLS playlist attribute and AVFoundation API for HLS Interstitials, please refer to the reference links under this session.

AirPlay Video offers built-in controls that appear on your iPhone in apps, on Lock Screen, and in Control Center so you can easily play, pause, fast-forward, rewind, and adjust the volume on your TV. With AirPlay, users can also use AirPlay receiver's remote to perform same remote control actions. With interstitials, oftentimes for contractual reasons, you'd want to prevent your subscribers from jumping over or skipping through the ads. HLS Interstitials has inbuilt support for user navigation restrictions.

Two restrictions are defined. Skip: skip restriction would prevent one from playing the ad at a rate different from what is desired. Jump restriction prevents from jumping out of the current ad. An example of this is, if the viewer attempts to seek across more than one interstitial, at least one interstitial shall be played before allowing the seek to complete.

Navigation restrictions are enforced at the player UI level, which is AVKit or custom application code, while playing on the device or AirPlay receiver while AirPlay is active. Navigation restrictions can be applied in following combinations: Skip, Jump, Skip & Jump. Similar to all other properties, interstitial navigation restrictions can be configured by using two methods. Server-driven: to configure navigation restrictions using server method, HLS Playlist must set X-RESTRICT attribute under the EXT-X-DATERANGE tag. Client-driven: in this method, the application code shall set the property restrictions of AVPlayerInterstitial object. Using one of the methods above, you can ensure that navigation restrictions are enforced on interstitial playback for local playback on Apple devices as well as for AirPlay session on AirPlay receivers.

AirPlay receivers are separate hardware. Receiver can be Apple or non-Apple device. That means all of the non-Apple receivers are not executing AVFoundation code. One of questions which may come to your mind can be, does the application need to make any code changes, like configure any specific property, to ensure navigation restrictions on interstitials are enforced by the AirPlay receivers? No code changes are required, provided that the navigation restrictions are configured by using one of the methods we discussed, that is server-driven, by setting X-RESTRICT attribute in the playlist, client-driven, by setting restrictions property of AVPlayerInterstitialEvent object in your application code. Let's start building a playlist to demonstrate how the skip restriction is applied using server method, that is HLS playlist.

We have playlist with the primary asset of ten-second duration. We'll update this playlist with back to back ads. This update will add two ads of ten-second duration to the playlist, which will start playing at five seconds into primary asset. As the start date is same on both the ads, these ads would be presented in the order they appeared in the playlist. That is, in this case, ad1 will play first. After the playback of the ad2, the player will resume playback of the primary asset where it left off.

Notice that X-RESTRICT attribute is set to skip on first interstitial ID, ad1. So skip navigation restriction shall be applied on the playback of first interstitial, that is ad1. However, if you notice no navigation restriction - that is X-RESTRICT attribute - is present in the second Interstitial ID, that is ad2. This setup gives flexibility in your ad offering.

Now let's look at the client-driven method. To simplify the example here, we are going to add just one ad event, that is ad1. We create AVPlayerInterstitialEvent with primary item and the start time of the event. Then we set different configuration options and set skip restriction on ad1event. Finally, we set the ad1event on the controller. Let's talk about how you can optimize AirPlay playback on the receivers. AirPlay is supported on wide range of receivers. These receivers include Apple TV, set-top boxes, media sticks, sound bars, smart speakers, and smart TVs.

AirPlay receivers can have different hardware, software, and media processing capabilities. Due to the difference in media processing capabilities, some AirPlay receivers may have transition delays when switching between media assets. One of the common reason of transition delays on some AirPlay receivers is the time taken to switch the decoders from one video format to another. Due to the difference in media processing capabilities on some receivers, this switch between the decoders may take time, which can introduce transition delays. For interstitial use case, an example of this can be, the primary asset is encoded with AVC while the interstitial asset is encoded with HEVC.

To reduce transition delays between primary and interstitial media asset on AirPlay receivers, it is recommended to keep matching media operating parameters between primary and interstitial media asset. For video asset, this includes video codec, frame-rate, aspect ratio. For audio asset, this includes audio codec, sampling frequency, resolution, channel layout. To ensure synchronization within the primary or interstitial content, all presentation units of media components, that is, audio, video, subtitle, must be accurately timestamped and aligned within specification of each other. For your customers that have a requirement to increase subscriber engagement based on the language of interstitial or ads, it is possible to specify matching audio and subtitle tracks for interstitials. If the interstitial and the primary asset have similar audio/subtitle tracks, automatic track selection will be applied while playing interstitials. This automatic selection is applied for local playback and Airplay sessions. No code changes is required for this. Let's discuss briefly what considerations you must take into account when testing AirPlay in your streaming application.

For testing AirPlay video in your streaming application, you need to build a test suite with AirPlay sender and AirPlay receiver devices. AirPlay video sender devices include iPhone, iPad, and Mac. AirPlay video receivers include Apple TV, Mac, and more than hundreds of non-Apple TV receivers and smart TVs. Each of these receivers differ with respect to hardware, software, and playback capabilities. When testing AirPlay with your streaming application, it's a good idea to start with Apple TV as the receiver of choice, but it is highly recommended to select at least one device from each class of non-Apple AirPlay receiver. You can further divide the receiver devices based on media processing capabilities, that is high-end and low-end receivers. For example, if your streaming service supports Dolby Vision, include an AirPlay receiver capable of Dolby Vision in your test suite when validating AirPlay playback from your application. In addition, be sure to include a low-end AirPlay receiver which only supports SDR and stereo AAC audio. Please note, even if your streaming application is targeted for only one range of sender device, for example, iPhone, it is strongly recommended to build a test suite containing high-end and low-end receivers. When selecting AirPlay receivers to test, refer to Apple’s growing list of commercially available devices in the references link of this presentation. We're going to look at an example VOD plan.

Service provider in this example wants to offer three service plans. Plan A: VOD service plan with ads in which play/pause are the only user operations allowed while playing the ads. Plan B: VOD service plan with no ads. Plan C: subscribers enrolled in this plan are able to skip ads once the initial ad has concluded. In this example, the service provider wants to have maximum configuration for interstitial support in the playlist on the server. Also, the service provider wants to use the same playlist for all the plans. We are going to use a playlist similar to the one we used to demonstrate skip restrictions with minor changes. In this playlist, an EXT-X-DATERANGE tag schedules two ads, that is ad1 and ad2, each of ten-second duration. The ads will start playing at five seconds into the primary asset. The player will play both the ads and then resume playback of the primary asset where it left off. Skip restriction is set on both the ad events. To support subscribers signed up for Plan A, that is VOD service with ads, no application code needs to be changed as the playlist already has all the rules for interstitials support. To support Plan B, that is no ads will be played, you can use the following AVPlayerItem property in the application code: automaticallyHandlesInterstitialEvents set to false. Setting this property to false will ignore AVPlayerInterstitialEvents specified by the server in the EXT-X-DATERANGE tags.

To support Plan C, that is, subscribers enrolled in this particular plan are able to skip ads once the initial ad has concluded, we need to make sure that restrictions on first ad, that is ad1, is retained. Modifications must be done in the client application code to override the restriction property set on the second interstitial that is ad2, such that skip restriction is removed.

The example code here demonstrates how application code can be used to override the navigation restriction set on ad2 in the HLS playlist. For this, we first extract AVPlayerInterstitialEvent for ad1 and ad2. We then create a copy of ad2 event and then clear navigation restrictions on the copy. We finally update the controller with the schedule of all the interstitials events, that is the original ad1Event and modified copy of ad2Event.

Now let's look at the demo videos to map the subscriber experience for all the plans. Each of the demos is pointing to the same HLS playlist, in which we have one main content, that is "chef decorating cookies," and two ads scheduled at five seconds into the main content.

We will start with the demo of example Plan A. We trigger AirPlay video.

This is the main content.

This is ad1 with the duration of 10 seconds. As the skip restriction is set by the server in HLS playlist, subscriber is unable to skip or fast-forward this ad. This is ad2, which also has skip restriction set due to this subscriber is unable to skip or fast-forward the ad.

The main content resumes from the offset ad1 was scheduled. Note, now the subscriber is able to jump forward.

For the second demo, we do similar steps as in the previous demo, that is, AirPlay to the TV, and we will demonstrate subscriber experience for Plan B, that is no ads will be played.

We are using the same HLS Playlist that we used for previous demo, but we are not going to see any ads scheduled at five seconds, as in the application code, we set the property automaticallyHandlesInterstitialEvents for the main item to false.

We now jump to our final demo to demonstrate subscriber experience for Plan C--that is, subscriber having the ability to skip ads once the initial ad has concluded. We start with the main content.

This is ad1 that will play at the regular rate, and subscriber is unable to skip or fast-forward this ad.

As we have programmatically cleared the restrictions set on second ad, subscriber will now be able to skip the ad2, that is a drone shot of Los Angeles.

To summarize this example, we demonstrated how the client application code can override the properties and configurations of server-side interstitials. Depending on the subscription plan, the client application code in the same app can let the AVPlayer schedule the interstitials and enforce properties set by server in HLS playlist, do complete override by disabling all the interstitials configured in HLS playlist on server, do partial override by updating specific property of interstitials.

To wrap up, you can schedule HLS interstitials by using the server method or in the client application code by using AVFoundation APIs. Application code can also override properties and configurations of server-side interstitials. To ensure navigation restrictions are applied for local playback and for AirPlay session, HLS Playlist must set X-RESTRICT attribute in EXT-X-DATERANGE tag, or the same can be achieved in the application code by setting restrictions property of AVPlayerInterstitialEvent object. To reduce transition delays between primary and interstitial media asset on AirPlay receivers, it is recommended to keep matching media operating parameters between the media assets. For testing of AirPlay in your streaming application, include wide range of receivers in your test suite. Hope you found this session informative. Thank you.