Explore HLS variants in AVFoundation

♪ Bass music playing ♪ Nishant Nelogal: Hello, and welcome to WWDC.

My name is Nishant, and I’m an engineer on the AVFoundation team.

Today, we will be looking at HLS Variants.

First, we will look at how you can inspect HLS variants, using AVFoundation APIs, and later on, we will see how we can use the HLS variants with downloads.

So let's start with variant inspection.

You all must be familiar with what a typical master playlist looks like.

Here is an example.

In this particular playlist, we have two variants.

One of the variants is an SDR variant with stereo audio, and the other one is a Dolby Vision variant with Dolby Atmos audio.

The asset representing this particular playlist may be presented something like this in your app.

Looking closely, you see that the asset has 4K, Dolby Vision, and Dolby Atmos badges.

Earlier, you had to get such information out of the band.

Now in iOS 15, you can directly inspect the HLS playlist to infer them.

For that, you start with an AVURLAsset pointing to the location of your master playlist.

Later on, you can obtain the HLS playlist via the variant's property.

This AVAssetVariant, as we see here, represents an HLS variant as is from the master playlist.

So it has multiple properties representing different media attributes.

Some of the attributes, such as the media bitrate, can be accessed directly.

Other attributes, such as those related to video and audio renditions, are grouped together in their own subclasses.

They are called as VideoAttributes and AudioAttributes, respectively.

As you see, each of them has relevant properties, which you can use to understand your asset.

Now you know how to inspect HLS variants in AVFoundation.

Let's see how we can use them with downloads.

Downloading HLS content for offline playback has been supported since 2016.

If you're not familiar with HLS download APIs, I would suggest you check out our earlier talk on the topic from WWDC 2020.

In iOS 15, we are taking our HLS download APIs and making them even more powerful.

Typically, you would like to influence the HLS variants selected for the download.

This may be due to business requirements, or you just want to provide more choice to your users.

Before, you could provide such input via downloadTask options.

We had an option for HDR, an option for lossless audio, and a few other attributes.

In iOS 15, we are expanding the variant selection with the use of NSPredicates.

You may be already familiar with using predicates from Core Data.

If not, don’t worry, you will learn about them today.

So to get started, let me introduce the variant qualifier interface.

This interface allows you to specify your variant preferences to AVFoundation.

And, as I said, they can be constructed using NSPredicates.

Let’s understand by looking at few examples.

Here, we have a NSPredicate which expresses peak bitrate to be less than five megabits.

You use this to construct your variant qualifier, which instructs AVFoundation to prefer variants under five megabits.

Simple enough, right? Let’s take a look at another example.

Here we create an NSPredicate for HDR video range.

And similarly as before, you can construct a variant qualifier for it.

You can also combine multiple predicates to create a compound predicate and use them to create your variant qualifier.

Any property on variant can be used to create a predicate against.

For properties such as audio channel count, which cannot be easily expressed using the predicate format string, we have custom constructors.

You can look them up in our header doc for variant qualifier.

Once you have your variant qualifier, you use it to create something called as content configuration.

Each content configuration represents a set of video, audio, and subtitle renditions.

OK, let's understand with an example.

Here is the variant qualifier which combines the two predicates we saw earlier.

Eh, this is compounding. Ha-ha.

It informs AVFoundation to prefer HDR variants under five megabits.

We also have a set of media selections representing English and French audio, and English subtitle renditions.

Both of these can be used to create a content configuration object.

You can create multiple of these content configurations and provide it to the downloadTask.

These multiple content configurations are tied together by the download configuration interface.

This is the root which holds everything together.

It is created with an AVURLAsset and needs an asset name, and optionally, an image.

The asset name and image are displayed in the Settings app.

This allows your users to manage all of their downloads in one place in the Settings app.

And of course, the downloadTask can be configured with multiple content configurations.

Looking closely, one of the content configurations will be designated as primary and the rest will be auxiliary.

The difference between the two is you typically want to download a primary set of video, audio, and subtitle renditions and then complement them with additional audio or subtitle renditions.

By specifying your additional renditions as auxiliary content configurations, you instruct AVFoundation to optimize and avoid downloading multiple video renditions.

It becomes more clear as we go over an example.

Here is the complete example.

First, we start by creating a download configuration with AVURLAsset and a title.

The primary content configuration is the same one we saw earlier.

It is configured to download HDR variants under five megabits, with English and French audio and English subtitle renditions.

In this particular example, we would like to complement it with an auxiliary content configuration to download English audio in lossless format.

Now we have both our content configurations, which we want to download.

Make sure to set optimizesAuxiliary ContentConfigurations to true.

By the way, it is true by default, and it allows AVFoundation to choose lossless variant, such that the lossless variant’s video rendition is the same as the primary content configuration.

Setting it to false may cause lossless variant to be evaluated independently, which may cause duplicate video renditions to be downloaded.

This may increase your download size, which we don’t want.

OK, once you have the download configuration, you can use it to create the downloadTask.

You resume the downloadTask to start the download.

Starting in iOS 15, you can also observe the progress of the downloadTask using NSProgress interface.

The NSProgress object is KVO-observable, and you can use it to update your user interface.

Moving on.

We understand that some of you may have business logic for selecting variants, which may be harder to express via predicates.

For such cases, you can also choose to choose the variants you want to download explicitly.

Here in this example, we have already picked our primary and auxiliary variant and the media selections to go with it.

We can instruct AVFoundation to download them as is, by creating a qualifier with the variant directly.

Be careful to select variants so that they are playable on the device where they are downloaded.

That's all we have today, folks.

To wrap up, we looked at how you can inspect HLS variants and use them to configure the downloadTask.

Along the way, we encountered different interfaces for downloadTask configuration.

The first one was variant qualifier.

This is used to express your variant preferences.

Then we saw content configuration interface.

The content configuration ties together your variant preferences and your media selection choices.

Finally, we saw the download configuration.

The download configuration is the root interface which ties everything together.

Last, but not the least, we also learned that you can monitor your downloadTask using NSProgress.

For more information, check our header docs.

They're very well commented.

Thank you for watching and bye-bye! ♪