Bring desktop class sync to iOS with FileProvider

Hi, I'm Johannes Fortmann from the Cloud FileProvider team.

Today we will talk about how to use the new FileProvider API introduced in iOS 16 to bring desktop class sync to iOS.

After the introduction, we'll do a quick recap of the aims of a file provider.

We will discuss the optimal approach for architecting your app, as well as best practices that become especially important on iOS.

Last, I'll show you a quick demo of a provider running on iOS.

Big Sur introduced a declarative API for syncing files to your Mac.

This API has been adopted by many cloud vendors to great success.

My team has been steadily working on improving the API, and we're excited to also make it available on iOS 16.

This API will enable your iOS apps to provide what we call "desktop class sync." What do I mean by that? As apps on iOS get more powerful, it becomes important for them to be able to access a shared location on the file system.

People want these powerful apps to be able to access all types of file system objects.

If they choose, their app should be able to access folders and create new files.

We want all of this to happen while guaranteeing consistency.

What does consistency mean? On iOS, background runtime has historically been limited because of power concerns.

At the same time, changes are expected to be uploaded in the background.

The modern FileProvider API was introduced to solve this problem.

At a basic level, you implement an app extension that is responsible for enumerating items, fetching and uploading contents, and updating the list of items if they change remotely.

The system is responsible for exposing the information you provide and for maintaining consistency.

An important task of the system is to keep track of errors and retry if necessary.

For some operations, such as fetching contents, retries come naturally.

The user is actively waiting for their download and probably monitoring the progress bar closely.

Uploads, on the other hand, require scheduling.

By tracking the state of the items on disk, the system ensures that the updated contents get uploaded.

Progress and errors are tracked to retry the upload if necessary.

Another complex topic is consistency during uploads.

The system manages clones of the file contents to ensure that during an upload subsequent accesses to the file succeed and show the correct data.

During these operations, the system also ensures that the local version stays consistent, even with multiple apps accessing it, and that includes sync downs from the remote server.

This is implemented transparently to you using APFS features and file coordination.

Storage limits are an important constraint on mobile devices.

The system uses an APFS feature to atomically track the change state of local files.

This allows it to transparently evict files that don't have local changes based on disk usage and least recently used status.

Files that are fully uploaded do not count against your app in the Storage Management pane in Settings.

You may have noticed that so far we have talked about the system and the extension.

Let's talk about where your app comes in.

I recommend that you employ a strict separation of concerns.

The system is responsible for managing the structures on-disk and scheduling tasks.

Your extension is responsible for performing those tasks to sync up and down.

The system tracks all state about the file hierarchy and which parts require sync.

That means that your extension can be very lightweight.

It does not have to track any item-specific state at all.

Your application is not responsible for any syncing.

Ideally, it doesn't have to talk to the server at all.

Instead, it interacts with your extension through two mechanisms.

It can interact with the extension indirectly, the same way that any other application on the system does.

There is API to fetch the file URL of any managed item, including the root.

These locations are then accessible using the regular file system APIs.

Alternatively, your app can request a direct XPC service connection to your extension.

This is particularly useful to handle tasks that cannot be expressed as file system manipulations, such as sharing files or resolving conflicts.

Both of these mechanisms can also be used by FileProvider UI extensions to provide an additional integration point in the Files app.

I would like to touch on three points that become especially important with stateless providers.

First, let's talk about uploads.

As I mentioned earlier, the system keeps track of uploads and will grant your extension time to perform the upload.

An important consequence of this is that you must keep the system aware that your upload is actually progressing by reporting progress.

If an upload task doesn't progress, it will be cancelled.

The system provides a grace period to wind down the upload cleanly, but if the cancellation takes too long your extension will be terminated.

Let's check out the code.

To implement a cancellation handler, simply set it on the progress return from the task-specific method.

In case of uploads, that's modifyItem.

In your handler, you cancel the actual upload work you were performing.

Of course, you'll also need to call the completion handler to signal that a cancellation error occurred.

The code example here uses an async task cancellation to make this convenient, but you could also call the completion handler manually.

Next up, let's talk about the sync down path.

When the user interacts with their files, your main app will not be running to receive changes from your server.

To still inform the system about remote changes, you should implement push notifications.

PushKit exposes a specific push type for file providers.

You can register for these pushes right from your extension.

On your server, you send pushes with a well-defined payload.

The system will receive the push and refresh the current state if appropriate.

As with the other types of tasks, the system may delay the actual refresh depending on the situation, like battery state or whether the user is currently looking at their files.

This last thing is just something I would like to call your attention to: the system manages the folder hierarchy that your extension reports.

This allows it to vend the entire folder hierarchy.

Your extension doesn't have to do anything extra here.

This is enabled by default for modern file providers.

Let's have a quick demonstration of what kind of workflows that last feature enables.

I've set up my device with this session's sample code.

We have ported our sample code to iOS.

We built an iOS app to handle logging into our server but the extension is mostly unchanged from the macOS version.

I'm running the sample code on my iPad right now.

I've got Files running on the right side and it's already synced my files.

I've also written an app that takes advantage of folder selection.

My app applies a sepia filter to all images in a folder.

This type of application benefits from folder access because it can operate on all items in the folder without forcing an interaction for each individual item.

With desktop class sync, I can simply drag a folder from the Files app into my batch editor.

Let me pull up the folder and files so that we can monitor the progress.

I then push the button and all my photos get downloaded and modified.

After modification, they are automatically uploaded.

The upload progress reported by the extension is surfaced to the user at the bottom of the Files app to keep them in the loop.

Let's say I wanted to implement something like this with my app.

First, let's implement dragging an item.

To allow initiating a drag, you implement the onDrag method.

The method will return an NSItemProvider.

You register a file representation on the itemProvider with the type of file you will be dragging.

In our case, that's a folder.

Use the getUserVisibleURL method to fetch the URL.

On the receiving side, implement onDrop to mark a view as the drop target.

You can then load the file URL from the appropriate item provider.

Note that this is going to be a file that lives outside your sandbox.

For your app to access it, it will have to consume and release the security scope of the URL.

What are your next steps? We've updated the sample code to include an iOS app.

Download it and experiment with setting up a simple stateless provider.

If you're starting from scratch, be sure to use the updated Xcode template.

It includes a basic frame to get you started.

To learn more about file providers and how to implement them, refer to "Sync files to the cloud with FileProvider on macOS" from WWDC21.

Thank you for watching.

I'm excited to use your performant and reliable providers on iOS devices.