Integrate your media app with HomePod

♪ ♪ Alex: Hi, I’m Alex Abramets.

Welcome to "Integrate your media app with HomePod." I am here to present an exciting, new Siri capability for the Home.

Starting with iOS 17, any HomePod owner will be able to ask Siri to play media using apps installed on their iPhone or iPad.

This new Siri ability will allow apps currently supporting SiriKit Media Intents to seamlessly play content on HomePod.

First, I am going to introduce expanded support for media apps to play on HomePod.

Then, I will cover supported use cases and experiences and show the best ways to implement them. Lastly, I will talk about how this capability integrates with the Home.

Let's get started by going over HomePod support for more Media apps.

Once HomePod processes a request, a SiriKit intent is sent to the iPhone, where Siri starts your app and AirPlays content back to the speaker.

We will talk more about AirPlay support later in this session.

To fulfill the request, HomePod will be communicating with iPhone over Wi-Fi, which means that devices need to be on the same network at the time of the request.

Devices don’t need to be in physical proximity.

If I am in a living room talking to my speaker and my phone is charging in the bedroom, HomePod will successfully play.

Siri requests to play media work with multiple destinations, including most speakers supporting AirPlay.

Any app supporting SiriKit Media Intents today will be able to use this capability with no additional changes.

Later, I will talk about a few things that can make the user experience and performance even better.

Requests to Play, Add, and Affinity requests for the apps not natively supported on HomePod, will be routed to the primary iPhone. Please note that on HomePod, requests to find or search for media will result in playback.

Siri will rely on voice recognition to route requests to the device of the recognized Home user.

In addition, requests to "like" or "add" songs will work even for content that was AirPlayed without using Siri. Now many existing media apps will be available on HomePod, and it’s not just music. Many types of media are supported. Music, audiobooks, podcasts, radio, meditation, and more.

Any app adding new support for SiriKit Media intents will be automatically supported on HomePod.

Everything your app can do on iOS today will work on HomePod. For example, app users can ask to play artists, albums, songs, music genres, and hits.

When the app name is used in the request, Siri will check if the app is installed on your iPhone.

If the app is installed, Siri will establish AirPlay connection and ask the app to start playback.

Maybe your app is about listening to Podcasts or Audiobooks.

Now it will be possible to catch up on the news or continue listening to your current audiobook on your HomePod.

Siri will understand requests to change speed, fast-forward, and control playback.

Requests specific to your app and not covered by Media intents can be implemented using App Intents.

Check out "Explore enhancements to App Intents" and other related sessions.

So how do I add Siri to my app? Let’s go over what is needed to support SiriKit Media Intents, then talk about making sure everything works reliably and provides a great experience. Apps with existing support for Media Intents will just work. For example, if SoundCloud is installed on your iPhone and you ask HomePod to play something on SoundCloud, media will play on the speaker. Adding playback support via Siri to your app includes several simple steps.

You’ll need to add an intents extension and implement a few methods to wire up Siri callbacks with search and playback.

You can find detailed instructions in "Introducing SiriKit Media Intents" video from 2019 and "Expand your SiriKit Media Intents to more platforms" from 2020.

Let’s inspect the code from a sample app that can play a song. Here, I am checking that the request was for an entity named "Push it" and, if so, return the media item. Let’s give it a try.

Hey Siri, play "Push It" by Dukes on Control Audio.

Siri: "Push It" now playing from your iPhone.

Alex: The song plays as expected! Now let’s ask something else.

Hey Siri, play jazz on Control Audio.

Siri: I couldn't find that in your Control Audio library. Alex: This time, Siri said it didn’t find anything.

This happened because the function returned an "unsupported" response since my app doesn’t support playing genres.

But I can make Siri respond better! I can provide a more specific reason for not being able to play, like "unsupported media type" or that the app requires a valid login.

For such cases, Siri will give responses that will aid in resolving the issue.

Now, I am going to change my code and add a guard statement that will return a more specific error code if the request was to play any music genre.

Hey Siri, play jazz on Control Audio. Siri: Control Audio doesn't support playing that.

Alex: So now, Siri says that the app doesn’t support playing that, which is a more helpful response.

You might be wondering what will your app receive when a Siri request is made? In the previous examples, we discussed the "mediaName" and "mediaType" fields.

Let’s review some of the most common intents that your app might get.

A simple "Play" with just the app name will send an intent without any search criteria.

Music apps could play a personal mix or a station. Podcasts and audiobook apps may resume the last played item.

For "Play music in app" requests, "music" translates into a "mediaType." "Play" request with entity name will provide a mediaName field only.

It will be up to the app to determine the type of the entity, whether that’s a song, artist, podcast or something else.

Mentioning "song" and its name will produce a mediaType equal to Song and a mediaName field.

This is how asking for "popular hits" will behave.

The media name will be set to "music" and the sortOrder will be "popular." Asking to play something next will tell the app to add it to the queue.

Notice that "playbackqueuelocation" has a value of "next." If your app supports news, podcasts or audiobooks, you may receive requests like "play my audiobook," "play the latest news," or "play tech news podcast." You should explore and experiment with the intents produced by Siri for the requests you expect to be sent to your app. After resolving the results, Siri will execute the "handle" step in your intents extension. You can return one of the two responses "handleInApp" and "continueInApp." "HandleInApp" will instruct the system to start the app and play in the background.

"ContinueInApp" will ask the system to start the app in the foreground.

For requests to play Audio, background playback will result in a better experience since it doesn’t require you to unlock the phone.

When a request is made to HomePod, it is even more pronounced, since your phone could be nowhere near.

Please keep in mind that your app’s responses to Siri callbacks should be reasonably fast.

Slow requests are often abandoned or simply time out. Make sure that your app responds in at most ten seconds.

It is preferable to be fast as possible, but after ten seconds, Siri's response will be generic and not very helpful.

What if you’d like to tell Siri that you like the song or a podcast, or wanted to add it to your playlist? Supporting INAddMediaIntent and INUpdateMediaAffinityIntent will allow your app to work with these requests.

The implementation is very similar to INPlayMediaIntent.

For AirPlayed content, Siri will send the request to the iPhone of the speaking person. Let’s go over a few things that can make the integration of your app with Siri more robust.

First up is personal app vocabulary.

Using personal app vocabulary, your app can inform the system of user-specific entities that requests might include.

These can include personal playlists, purchased audiobooks, podcasts, favorite artists, or authors. Normally, only foreground apps can start playback, but during Siri request, your app can start audio in the background.

For this to work, it is very important to have the audio session properly configured prior to activating that session.

This applies to all supported platforms, including HomePod, iOS and CarPlay.

In this example, by setting the audio category to playback, the app will inform the system that it will be playing audio. Audio category, mode, and route sharing policy will affect how the system handles audio playback and interruptions. Setting the mode to "spoken audio" will pause the playback when interrupted instead of temporarily reducing the volume of your content.

This is especially useful for apps that play podcasts or audiobooks, so that listeners don’t miss any important parts of the content. Let’s check out what happens if the audio category is not set to ".playback" prior to activating the session.

Hey, Siri, play "Push It" by Dukes on Control Audio.

Siri: "Push It" now playing from your iPhone.

Alex: While the app is in the foreground, the audio plays, but once the app goes into the background, for example when I lock the phone, playback stops.

It works, but not the way you would expect it to. Let’s find out what happens if I change my code to correctly configure the audio session.

Hey, Siri, play "Push It" by Dukes on Control Audio.

Siri: Here's "Push It" from your iPhone. Alex: When I lock the phone, the audio continues playing.

♪ ♪ HomePod and iPhone can do many things, so it is not uncommon for the audio playback from your app to be interrupted by an incoming call, navigation prompt, Siri requests, or other apps playing something else.

It is important to handle interruptions so that your app can continue playing, especially when someone across the room is talking to HomePod and doesn’t have their iPhone handy.

Ok, let’s talk about AirPlay.

For the system to successfully AirPlay content from your app, make sure that you do the following: set the audio category, mode, and route sharing policy appropriately as I talked about earlier, use MPNowPlayingInfoCenter to inform the system about the currently playing item, and MPRemoteCommandCenter to receive remote commands, like play, pause, and skip.

These steps will get the media from your app playing on AirPlay speakers.

But since the audio data is being delivered to another device, the system needs a buffer and will accumulate some data to ensure smooth playback.

This may introduce a delay each time media is resumed or stopped. Using one of the APIs for buffered playback will remove this concern.

Check out this year's session called "Tune up your AirPlay experience" for more on this topic.

Next, I will talk about additional considerations for this capability. What if I have multiple iPhones or iPads? How will Siri know which one to use? The answer is that Siri will route SiriKit Media requests to the primary iPhone, which is the one that is set up to share your location under Apple ID and Find My settings.

Since Siri will communicate with other devices to fulfill these requests, the requesting user will need to be registered in the Home and have Recognize My Voice enabled. You can find this setting in the Home app.

Enabling Personal Requests is not required for this feature to work.

Keep in mind that if you don’t have "Recognize My Voice" enabled or you are not registered in the Home, you will not be able to play media from your or anyone else’s devices.

If you just installed the app or never asked Siri to play from it, the first request to Play using the app will direct you to your iOS device and request permission for Siri to access the app data. This permission is only needed for the first Siri request to Play from your app. Before I let you go, here are the things you should remember.

Adding SiriKit Media intents support is the one thing that is required for your app to work on HomePod.

You can further improve the user experience by adding Buffered Playback, supporting more complex queries, and providing Siri with more context.

Thank you! I cannot wait to see your apps working with HomePod! ♪ ♪