Share files with SharePlay

♪ ♪ Adam: Hi. My name is Adam, and I’m an engineer on the SharePlay team.

Today we’re going to be talking about an exciting new feature that many of you have been asking for.

As you may have guessed by the session title, we’re going to talk about sharing files over SharePlay.

First we’ll talk about what transferring attachments means.

We’ll refer to files, or any large amount of data that you want to share, as “attachments." Then we’ll talk about the new API in GroupActivities, the GroupSessionJournal. After that, we’ll talk about late joiners, and finally, we’ll walk through and see how easy it is to adopt the GroupSessionJournal in our DrawTogether app.

Let's start with what transferring attachments means.

Let’s talk through our drawing app.

We have it so that we can sync drawings between two devices with the GroupSessionMessenger.

But what happens if we drop an image into the canvas? Well, because of the size restriction on the GroupSessionMessenger, you weren’t able to utilize SharePlay for this before.

But now, we’ve made it so the same Apple infrastructure that gives you super fast messaging between devices can now transfer attachments as well. So now when you drop a cute picture of your dog into the canvas, it transfers through our cloud infrastructure faster than ever. This utilizes multiple layers of Apple technology in order to minimize the data needed to be transferred and minimize the idle waiting time, all to make the experience as fast as possible and all of that happens at a system layer.

But that isn’t it.

SharePlay is built with user privacy in mind so all of this is done while making sure that the data you’re transferring is end-to-end encrypted between devices.

Apple does not know about the content of the data transferred. And this isn’t just for cute dog pictures of Mochi, it can be for anything you want! Have a file like a PDF? Go for it! How about a voice recording or annotation? Any user content that is being collaborated on can now be faster than ever powered by this new API.

So, now what does the API look like? Well, let’s dive right in! Our main object to familiarize yourself with is the "GroupSessionJournal." This object is new to iOS 17. Just like you’re used to with the "GroupSessionMessenger," it’s initialized with your "GroupSession." But it’s important to understand the purpose of this object.

It’s an object that stays consistent for everyone across the GroupSession. This means that actions you take on the journal affect everyone and properties on the journal are similarly synced for everyone. Let’s walk through some examples.

Once you have your Journal, you can observe attachment mutations via the "attachments" AsyncSequence.

And you can upload any of your custom attachments with our "add" function. All you have to do is make your type conform to the Transferable protocol! So now, when one device calls "add," not only will their "attachments" AsyncSequence give the new attachment, but everyone on the GroupSession will see the same event happen and be able to load the attachment that you’re uploading! Likewise, if you were to remove the attachment by calling the "remove" function, then everyone's "attachments" AsyncSequence will be updated for the event.

And that’s pretty much it. You can see how easily you can upload and receive attachments over this new API.

Before we dive into implementing this in our DrawTogether app, let’s talk about some big considerations when utilizing this new API.

And it’s that attachments aren’t allowed to be infinite in size, they’re actually limited to 100MB. This is done to ensure that we think carefully about the user experience.

These experiences need to be fast and devices can only upload and download data so fast. Therefore, you should always try and minimize the size of attachments being sent.

That may mean that if you’re a drawing app, you restrict the photos allowed on your canvas to be less than or equal to 100MB. Any data oriented around user-generated content is perfect here, but larger attachments that aren’t user generated, like a movie, should still be served from your own servers and not through a user’s device.

Then there’s the life-cycle.

Anything put on the Journal will stay around for as long as there’s someone in the GroupSession.

This means even if the person who adds an attachment to the Journal disconnects from the GroupSession, the attachment will stick around.

But if everyone leaves the GroupSession, the attachment is removed along with them.

If you’re used to integrating with the GroupSessionMessenger you may remember this topic: Late joiners. Typically when a new person joins the GroupSession, your application would have to re-transmit your state of the world to that person so that they’re caught-up with the rest of the group.

This is managed by your application adopting GroupSessionMessenger and involves each device sending a message to the person needing catch-up, which can be pretty expensive.

Let’s take the following example.

Say that we have two people in a GroupSession. There’s Adam and Brian. Meanwhile Clara hasn’t joined yet.

So, before Clara has joined, Adam and Brian are having fun drawing.

Adam goes ahead and draws a dog bone and Brian gets it updated on his side too.

Now, Clara goes ahead and joins, but her canvas is empty when she joins! Her device needs to get the information that Adam and Brian were doing so that she’s caught-up with them. At this point both Adam and Brian will have to send Clara what they understand their state of the world to be and Clara’s device will converge so that she’s at the same point. Now if you add an attachment in-place of those drawings those messages sent from Adam and Brian can get pretty big.

As you can imagine, this becomes a problem. You’d be having each person re-upload 100MB just to give one device the image.

Well, now late-joiners are handled for you on the GroupSessionJournal.

Late-joiners will receive the uploaded attachments on their Journal without any re-uploads happening from others which makes for a faster and less bandwidth-extensive experience. Now that was a lot of information, so let’s jump into the fun part and see the magic at work and walk through how we would adopt the GroupSessionJournal in our DrawTogether app. This is our DrawTogether app that we’re gonna be modifying. As you can see, we’ve got some buttons here at the bottom that let us start a SharePlay session and a trash button to reset the session.

I think it’d be pretty nice if we had a button here at the bottom that allows us to select a photo we want to put on the canvas.

SwiftUI makes this easy to get started, so let’s jump straight in! There are four main components that we need to modify.

First, there’s our "Canvas" model.

This is our main model object for the whole app.

It keeps track of all of our strokes for both the local and remote participants and is our interface to GroupActivities. Then we’ve got our three views. There’s "ContentView" which has all subviews for our other two views, the "ControlBar" view, which maintains the buttons at the bottom that we saw earlier, and the "CanvasView," which draws all the objects from our model for strokes, and soon, images too.

Let’s start with our model.

We need this to track two things. First, all the images we get from the journal, which we’ll track with a new struct, "CanvasImage." Second, the locally selected image, if we’re trying to place one, and all the images.

Now let’s make the Canvas create a "GroupSessionJournal" And handle all the attachments from it which we’ll do in our configureGroupSession function.

And we’ll make a function to add the locally selected image to the journal that we can call from our views.

Now we’ve got all the non-UI pieces, let’s go ahead and modify some views.

We’ll start with the button to select a photo.

If you remember from earlier, we wanted to add this button in the ControlBar. Let’s go ahead and import PhotosUI and add a new button using the "PhotosPicker" view and put it straight onto the Model with "selectedImageData." We’ve gone ahead and created a simple view called PhotoPlacementView which visualizes the photo that was selected and allows us to move it to where we want on the canvas.

Let’s go ahead and add that view to the ContentView now.

Let’s see our progress.

We can start a session and then we get our Photos button.

Let’s select a photo, and bam! We have an image we can drag around.

Now the last part is us modifying the "Canvas" view to visualize all the images from the model.

Now if we try it out, we can see that the image appears on both sides as soon as we select the image’s final spot.

Phew. Now we’re done and I think we’ve got a pretty cool experience.

Thank you all for tuning in. Now that you’ve watched this session, go ahead and see what kind of amazing experiences you can now build with GroupActivities.

If you haven’t adopted GroupActivities yet, go check out “Add SharePlay to your app” as an intro to what SharePlay means for apps. And if you want to see how we got started with this DrawTogether app or see some examples of how to adopt GroupActivities, check out “Build custom experiences with GroupActivities." Thanks again, and we can’t wait to see what you build! ♪ ♪