Expand your SiriKit Media Intents to more platforms

Hello, and welcome to WWDC.

Hi. I'm Danny Mandel, and welcome to "Expand Your SiriKit Media Intents to More Platforms." We've got an exciting mix of SiriKit Media new features and experiences for you this year. We've got a lot to talk about. We're putting a particular emphasis on SiriKit Media Intents in the home this year. As part of that, we'll be opening up to new platforms. Additionally, we have some new features to discuss.

Finally, we'll finish up with some new ways to make your SiriKit Media experience even faster. So let's dive right in to the new SiriKit Media platforms. Last year, we made a great experience for you on the go, and this year we're bringing that same great experience to one of the places you listen to music the most-- in the home.

On HomePod, we're using the same media intents and a new cloud playback API to take advantage of the unique characteristics of the HomePod in the Apple Home. You'll want to check out the full developer program details at developer.apple.com/siri.

We're happy to announce that SiriKit Media Intents also now work on Apple TV. In addition to your iOS apps, we know many of you have custom Apple TV experiences, and we wanted to open up SiriKit Media to those apps as well. Now I'll hand it over to my colleague Ryan Klems to give us a demo of the new Apple TV functionality in action. Last year, in iOS 13 and watchOS 6, we introduced SiriKit Media Intent handling for your media applications. This year, we're happy to announce that we've brought that same functionality to tvOS 14. Now you can add Siri support to your applications on the big screen. SiriKit Media Intents allow you to add a wide variety of natural language queries to your application. You can do simple searches: Play music on ControlAudio.

You can ask to play a band: Play the band Khruangbin on ControlAudio.

You can ask to play a song by an artist: Play the song "Be Kind" by Marshmello and Halsey on ControlAudio...

and many more. So how do you do that? Adding SiriKit Media Intent handling to your tvOS application is done the same way you would add the support to iOS. Please see the 2019 WWDC session, "Introducing SiriKit Media Intents," for more information on the steps to add the extension target to your application. Now, let's do a quick recap of how we handle natural language requests in SiriKit Media apps.

The intent handler is the key class in handling SiriKit Media requests, and the key method to implement is resolveMediaItems. resolveMediaItems is where you evaluate the INPlayMediaIntent object that is received by your application and perform a search to turn it into the concrete media item that you want your app to play. After you resolve the media item, your handler will be called, and this will cause your application to be launched so you can begin playback. One difference with tvOS is that the customer is generally only interacting with one application at a time, and as a result, a foreground app launch is likely the more preferred interaction. Unlike on iOS where a background launch is more likely the preferred interaction.

You control this by returning continueInApp from your handle method instead of handleInApp. So, with the same few short steps that we showed you last year, you can add the ability for your application to play audio content through Siri on tvOS. We were very excited to see all the great adoptions of SiriKit Media Intents last year on iOS and watchOS and are looking forward to seeing what you do on tvOS this year. Thanks for that awesome demo, Ryan. Now I want to switch from platforms to new features that make the experience of asking Siri to listen to music even easier. As we work through these examples, we'll be using our example app, ControlAudio, which was introduced at WWDC19. So let's talk about the new Siri alternatives UI.

So, let's take a look at the current SiriKit Media experience in the new compact Siri UI. We ask to play a song from the album New Me, Same Us. ControlAudio chose a single song to play, and playback begins. There isn't a way to present more information than this, so we're only able to play that one song. We all know that sometimes we want something slightly different than what starts playing. So we've made it easier to make tweaks on the fly, without having to open the app and disrupt whatever you're doing while on the go.

We do so with the new alternatives UI in SiriKit Media. We can see that someone asked to play that same song, and Siri started playing it for them.

However, we can now also see this new menu that says "Maybe You Wanted," and if we tap on it, we are taken to the following UI.

And as you can see here, there are a number of different songs from the same album that someone might have wanted instead of the one that ControlAudio chose. Now we can tap on any one of those options to start playing that instead of the first choice. And that is the new SiriKit Media Alternatives UI. So how do we make this work? It ends up being quite simple.

In your current resolveMediaItems method, you're probably calling the success with resolvedMediaItem method on INMediaItemResolutionResult. Now, that method is going to return a single media item resolution result.

In order to return multiple items, all you need to do is switch over to the new plural version, successes with resolvedMediaItems, and anything after the first item in the list will show up as an alternative.

So the next question you may ask is how do you handle the taps when someone chooses one of the alternatives? All that happens when someone taps on one of the alternatives is that the media item they've tapped on is set as the media item in the INPlayMediaIntent that is passed to handle. What this means is that you can handle this just as you handle any other INPlayMediaIntent today.

Let's see the code change we need to make in order to get SiriKit Media alternatives working. The first thing we're gonna do is open our intent handler in our sample project, ControlAudio.

And we'll navigate to our resolveMediaItems method. Currently, we're calling the singular version of the method success with: So let's run the app and see what it does.

Play a song from New Me, Same Us on ControlAudio.

Here's "Where You Belong" by Little Dragon on ControlAudio. All right, we can see that it started playback, but there are no alternatives.

So let's go back to Xcode, kill the process, and update the method to return multiple media items instead of just a single one. So now we're calling the plural version of the method successes with: Let's run the app and see what it does this time.

Play a song from

"Where You Belong" by Little Dragon now playing on ControlAudio. All right now we have alternatives. Let's go play one.

And I'll play "New Fiction." And we can see that it started playing. It's just that easy to get the new alternative support hooked up in SiriKit Media. Now let's talk about some performance improvements that are new to SiriKit Media this year.

Last year, when we launched SiriKit Media Intents, we required the use of an intents extension to adopt your SiriKit Media functionality.

This year, there's a new way to handle SiriKit Media Intents-- in-app intent handling.

In-app intent handling moves all the intent handling stages into your app rather than in an extension. An advantage here is that it avoids a process launch, as you don't need to launch your intents extension and your app to start a background audio session. You only need to launch your app.

However, because it's launching your full app, this could potentially mean a slower Siri response time during the resolve phase. Intents extensions are small, lightweight processes optimized for quick launching, so you will definitely need to tune your full app launch experience to get the same quick response out of it.

One other benefit you could see is that you can start the player warming phase much earlier in your app, since you'll be implementing resolve there. So, if you know you need to kick off network requests to fetch credentials, you can start that at the beginning of resolve and ensure that the player is 100% initialized by the time you get to handle the intent. I refer you to the presentation, "Integrate Intents with Your App," to get the full details on in-app intent handling.

For those of you with existing extension-based implementations, we also have a performance boost this year via app pre-warming.

Looking at your existing SiriKit Media implementation, we go through the standard resolve, confirm and handle phases, and then once all those are complete, we issue a background app launch for your app to begin playback. This does provide for a nice separation of concerns in that the extension can contain all the intent handling logic, and the app can handle the playback logic. However, it can delay important app concerns such as playback engine setup or credential fetching until much later than they need to be.

With app pre-warming, SiriKit will get your app fired up earlier and have playback ready to go once your extension is complete. It's important to note that app pre-warming does require some additional work in your app, so you'll want to work with us to make sure you get it working properly. And last, there are benefits and drawbacks to both in-app intent handling and app pre-warming, so you're going to need to evaluate the pros and cons for your particular implementation to see what works best for you. We wanted to let everyone in the developer community know that, as the saying goes, we're eating our own dog food.

We're thrilled to have a common framework for our bug fixes, new features, and performance improvements that both you and the Siri team at Apple can benefit from. Thank you, and we hope you enjoy giving people great, new SiriKit Media experiences this year.