Improve access to Photos in your app

♪ ♪ Hi. My name is Justin. I'm a Photos engineer. Together with my colleague, Adam, we will talk about ways to improve access to Photos in your app. First, I will talk about the improvements we made to the system Photos picker. Then, Adam will show the new cloud identifier APIs and how you can use them to identify the same asset across different devices. Lastly, he will also discuss changes to the limited library access mode.

Let's start with the Photos picker. In iOS 14, we introduced PHPicker, the modern replacement for UIImagePickerController. The PHPicker API ensures privacy while providing your app the features you need, like multiple image selection and search.

If you haven't heard of PHPicker, you can watch our WWDC 2020 "Meet the new Photos picker" session to learn more about it.

In iOS 15, the system Photos picker has four major improvements. First, privacy. We believe it's important to let people know what information your app can access, which increases trust. In iOS 14, some people may assume that your app has access to their entire photo library, even if it only presents the out-of-process Photos picker and only has access to photos they selected. To help people understand the difference, we added a new section to Settings > Privacy > Photos. The new section contains apps that only use the system Photos picker, explaining that those apps only have access to photos people selected. We really encourage you to replace your custom picker with the system one if you haven't done so already. Next, let's talk about ordered selection. Sometimes, people may want to control the order of photos when selecting them. In iOS 15, your app can configure the picker to show the selection order, as shown here.

By default, we will still show the selection checkmark. To opt in to the new ordered selection behavior, you only need to enable multiple selection and set "selection" to "ordered." The PHPicker API works great when all someone needs to do is to select new photos. But what if people also want to deselect photos using the picker? In iOS 15, a new preselection API is available for you to use.

But before we talk about the new API, let's revisit how the PHPicker API is used currently. You just need to create a PHPickerConfiguration object and use it to initialize the picker.

When the picker session is completed, selected photos will be returned to your app. Suppose the configuration is initialized with a PHPhotoLibrary object, picker results will contain both item providers and asset identifiers. Your app can use the item provider to load and display selected photos.

In iOS 15, you can also present the picker with some photos initially selected. It allows people to select more photos or deselect those photos preselected by your app.

To do it, you need to create a new configuration object. You can give the configuration an array of asset identifiers you retrieved previously, ID: 1 and ID: 2 in this case.

When a new PHPickerViewController is presented, photos with ID: 1 and ID: 2 will be selected by default. People can deselect them or select additional photos.

You still want to keep the old picker results for now because you will need to use them later.

Let's say the photo with ID: 2 is deselected, but a new photo with ID: 3 is added to the selection.

When the picker session is completed, selected photos will be returned. But the first result only has the asset identifier without the actual asset data. Why? Because it was selected by your app. All results not selected by people using your app have empty item providers.

Old picker results have actual asset data, so you can use the old ID: 1 result to replace the one with the empty item provider. Because ID: 2 is no longer selected, you can discard the old ID: 2 result.

OK. Now we can continue handling selected photos like before. To summarize: if a picker session is completed, all selected assets, including preselected assets that haven't been deselected, will be returned. And keep in mind, returned item providers for preselected assets will be empty.

In the case a picker session is canceled, only preselected assets will be returned. All item providers will be empty.

All right, let's take a look at the code. To set preselected assets, you only need to initialize the configuration with a photoLibrary object and enable multiple selection. You can set preselected assets using the new preselectedAssetIdentifiers API.

In the delegate callback, you need to retrieve the existingSelection from the previous picker session. Then, you can iterate through all of the newly returned picker results. You should replace new results with the old results if they are part of the existing selection. Finally, you can save the updated selection.

Last but not least, let's talk about progress reporting. It can take some time to download a large asset if it's not available locally on the device, which can happen if iCloud Photos and Optimize Storage are turned on. Previously, you can only show a spinner while loading, which is not always the ideal user experience. I'm happy to announce that in iOS 15, your app can retrieve the actual loading progress from the picker.

The loading progress is available via the existing NSItemProvider API. You can use the returned progress to show a loading UI accordingly. That's all for the Photos picker update. Now I will hand over to my colleague, Adam, to talk more about the new cloudIdentifier APIs.

Thanks, Justin. With the new features and APIs we've added, Photos picker is the best way for most developers to use Photos in their apps. But we know there are special categories of applications that require a deeper level of access and integration for managing or editing photos, custom cameras, or apps that give people a way to browse their photo library in a unique way.

For those apps, PhotoKit provides a rich set of APIs for accessing and updating photos, videos, and albums stored in the photo library.

Assets and albums are given unique identifiers by PhotoKit that can be saved in your app and then later used to retrieve the same records from the photo library.

Every photo library and its identifiers are specific to the device they are running on, even when those devices are syncing libraries with iCloud Photos. So let's turn our attention back to your app. People love the way they can use apps on an iPhone when they're out and about and then, back at their desk, use a laptop or iPad to take advantage of the larger format. Your customers want a seamless experience when switching from one device to another. For example, let's say I've built a photo journaling app for people to document experiences with pictures and comments. It's perfect as an iPhone app for capturing content on the go. They should be able to pick up working on that journal right where they left off when they open my app on an iPad or Mac, with all the photos and content they added on their phone. So how does that work? How do you find the same assets when each device has its own photo library and distinct identifiers? It works by using the new cloud identifier APIs. They give you a way to find the same assets and albums between devices. You use cloud identifiers to look up the local identifiers that are unique to each photo library. And these new APIs are available on every platform that supports PhotoKit: iOS, iPadOS, macOS, and tvOS.

The goal is to give you a simple way to map assets between devices without having to worry about the cloud. It's designed to work best when running on an account signed in to iCloud Photos, but cloud identifiers will work even if the account is signed out or running on a system that has never been signed in to iCloud. It's all based on identifier mapping, so let's be clear. There are two kinds of identifiers. Local identifiers are what I talked about earlier, the identifiers you use on the device that are specific to that library. If you've used PhotoKit before, you've seen them called localIdentifiers in the API. Cloud identifiers are the new API, which are used to look up local identifiers on one device or another. They are represented by a newly available class called PHCloudIdentifier. Let's dig in to how to use them. We'll start on the source device, where your app has gathered a list of local identifiers for some assets. From that array of local identifiers, we want to get the corresponding cloud identifiers. Here's an example of how to use the API to get them. This call to the photo library returns a dictionary with cloudIdentifierMapping objects for each localIdentifier. We'll iterate through that dictionary and get a cloudIdentifier from the cloudMapping object. I'll talk about error handling in a bit. For now, we should have our cloud identifiers and we're ready to use them. And the biggest benefit of cloud identifiers is using them everywhere, so we need to share them to other cloud-connected devices.

You want to make them available to your app when it runs on any device connected to this iCloud account. You can archive cloud identifiers to a string and share that data using CloudKit or another network data service. How you package up this data is really up to you and your app's needs. Think about how you'll use the information on a second device. How do you reconstruct the application state to include the content referenced by those cloud identifiers? Now, on a second device, once you've downloaded and unarchived the cloud identifiers, we want to use them to look up the library-specific local identifiers.

Working in the other direction, this call to the photo library returns a dictionary with localIdentifierMapping objects for each cloudIdentifier.

Again, we'll iterate through the dictionary and this time get a localIdentifier from the localMapping object. Once I have all the localIdentifiers, I can use them to fetch and display the assets I had on the source device. You can see how the API makes it easy to look up identifiers in both directions, but the reality of different device configurations, account settings, network states, and countless other variables can mean things aren't always that simple. To handle that complexity, the API uses a pair of helper objects to hold a valid identifier or an error indicating why the mapping failed.

There are two kinds of errors you should handle when looking up identifier mappings. If the photo library isn't able to resolve an identifier mapping because the underlying record isn't present or the app doesn't have access to it, the mapping object returned by the API will contain the Identifier Not Found error.

Here you can see how we're getting the error code from the localIdentifierMapping object if the identifier isn't set. For an Identifier Not Found error, it makes sense to keep track of the unresolved cloudIdentifier so we can put a placeholder in the UI to indicate the photo couldn't be found on this device. The other type of error happens if we can't resolve an identifier mapping uniquely. In other words, there are multiple assets that match the provided cloud identifier. This can happen if the cloud state isn't totally in sync on this device and the library has to rely on the image content to find a match. In this case, the mapping object will have a Multiple Identifiers Found error, and the error user info will contain the matching identifiers.

With the Multiple Identifiers Found error code, we can get the matching local identifiers from the error user info using the LocalIdentifiersErrorKey. In this example, we've stashed them in a dictionary so we can later fetch and show the matching assets and let the customer decide.

That's an overview of cloud identifiers and how to use them. Keep in mind that looking up the mappings between local and cloud identifiers can be expensive, so use local identifiers in your app interactions with the photo library, perform the mapping work at load and save points, and then store and share the cloud identifiers to make them available on other devices. Next up, limited library.

Before we dive into the updates to limited library, let's quickly review how photo library access works. When someone launches an app and requests access to photos data using PhotoKit, the system requires their permission before granting the app access. And the first time this happens, they'll see an alert like this one. This is really important to our customers. It gives them the control over how much access to give this application: to allow access to a limited selection of photos, full access to their entire library, or to deny access completely. If they choose the Select Photos option, then the app is granted limited library access.

You can refer to our WWDC '20 session "Handle the Limited Photos Library in your app" for a more complete description of limited library mode and how to design your app to work best with it. For now, it's enough to know that limited library access is designed to be transparent to the app developer. In general, PhotoKit APIs should work like normal, behaving like the library only contains the photos that were selected. It works conveniently with common workflows. For example, when an app adds a photo to the library, that photo is automatically included in the app's limited library selection.

There is another workflow we see in apps using a custom album to keep track of the assets they've added. But in iOS 14, limited library mode did not allow creation or access to custom albums.

I'm happy to tell you that in iOS 15, we've added support for applications to create, fetch, and update their own albums when running in limited library mode.

Now let's turn our attention to a different workflow, and for this, I'm going to refer back to my example photo journaling app. Since I want to give my customers a great experience when they've chosen limited library access for my app, I've customized the interface to present limited library picker as part of creating a new journal.

This provides a natural step during journal creation for them to choose more photos. But once the selection is updated, it would be great to know what photos were added, since they're the ones my customer wants to use in this journal. With a new presentLimitedLibraryPicker API in iOS 15, I can pass in a completion handler that gives me the identifiers of the photos they just selected in the picker. This lets me know when the selection update is done and which assets they added.

That covers our updates for limited library, support for apps to create and work with their own albums, and a new API to give apps a way to track what was added to the limited library selection.

That's most of what we had to talk about today, but before we go, I'd like to leave you with the following. For those who haven't updated your code in a while, it's time to move away from Assets Library. It's been deprecated since iOS 9, and we're planning to remove it in a future SDK. This is the time to move to the new Photos picker or PhotoKit for a deeper integration with photos. And one more reason to make the move to PhotoKit: we know we've had long-standing gaps in error handling, and I'm so happy to tell you that we've finally addressed those with new error codes to identify the underlying issue during change request processing, resource requests, and library access calls.

We can't wait for you to check out all the great new features we've added to Photos picker and PhotoKit in iOS 15. Thank you.