Production expectations for Apple Immersive Video: Learnings from “Submerged”

Hi everyone. Hi everyone. Streaming at home.

I am Alex and I am one of our producers working across many of our immersive projects, and one of the other exciting pieces we released last year was our first scripted project, Submerged. Let's take a quick look at what we made.

Now. Everything we're doing here is the first time that anyone's ever doing it.

When you put on the Apple Vision Pro, it does change the way you think about creating a story.

It's a wonderful new medium that just expands the horizon of storytelling.

Because you're not watching a movie anymore.

You're inside the story.

It's going to change the future of filmmaking.

All right, well, that was super exciting.

As you just saw, Submerged brings viewers into a World War Two submarine and follows its crew as they struggle to combat a frightening attack. We filmed this 17 minute piece in three international locations with just over nine shoot days. And as Edward Berger, our incredible director, mentioned in the previous clip, there were a lot of firsts on this production.

Not only was it our first scripted piece, but it was the first time we collaborated with a large, critically acclaimed film crew.

It was the first time we deployed multiple Vision Pro devices on set to be used by tools, as by the production team.

It was our first time filming large special effects stunts and successfully filming in low light and really tight spaces, and we often get asked the question, what's different? How is this production different than traditional 2D filmmaking? And the answer is there's not as much of a difference as you would expect. So this is the production crew in Brussels. They're working on the sets that would be fully submerged in a water tank, which is pretty epic. And we learned with this production you can have traditional crew positions like your art department, your camera operators, your gaffers. You can have similar budgets in filming schedules. But because of this new technology, what's different is really the how as a producer, how are you allocating your resources and how are you building in the right amount of time to execute a successful production.

So today we're going to take a look at a few different areas of production where this new technology changes how you spend your time and your resources. And we're going to begin with the development and pre-production process. But first, let's quickly go over what a traditional production process looks like.

So we usually start with the creative development first emerged. This is where the script writing began. Then we moved into pre-production. We found our locations in Prague, in Brussels and Malta. We hired our crews. We began building our submarine. We casted the film and we started camera testing. Then we moved into production. The team flew overseas and they hunkered down for the shoot. And look, this is really where everything came together. We saw the finished set that would get flooded with water. We saw the actors in their World War Two costumes, the fight choreography. It was really exciting. And then we hit post-production where we put the footage together in the edit. We had visual effects teams building the outside of the submarine, and we added unnerving sound design elements like the slow drip of a leaky pipe. And then finally we hit delivery, where the piece landed on platform.

But it's really here in the development and the pre-production phases where we've learned we need to spend more time and more resources. We've learned we need to account for plenty of time to get our crew up to speed and comfortable with this new tech, which meant camera workshops and lots of camera testing like you see here.

We also conducted a lot of demos to make sure our crew spent lots of time in the Vision Pro before production, so everyone felt confident using it. Because when you're on set and you're in a fast paced filming environment, not understanding the basics of navigating around a Vision Pro to review footage can eat up precious amounts of time during the day. Another big part of that development phase is previs. Like Anton spoke about, previs became crucial for us producers so we could understand what filming tools we would need to use based on the camera positions and the blocking. But also, previs helped us figure out how we were going to build our environment. And that leads us to the second area we realized was greatly impacted by bringing this new technology into the production process, and that's set design and lighting.

Building a period accurate, accurate World War Two submarine from scratch was no small feat. And the production's incredible art department, they worked around the clock to bring this vision to life. And without previs and extensive camera testing like you see here. So this was a mock up of the submarine corridor. We wouldn't have realized that the actual sub design, even though it was period accurate in size, was actually too small for our immersive camera and could potentially cause viewing discomfort because of the proximity to the walls on either side. We also learned we had to pay Extreme attention to the detail in the set design because the immersive camera, it mimics the human eyes, its acuity, or the sharpness of the image would pick up any inauthentic detail. So, for instance, the art department, they had to source and create all of the dials around the submarines with the right materials. And trust me, there were a lot of dials you couldn't get away with just using wood or plastic. And so for the producers, that meant that we had to build in more time and allocate more resources to set design in authentic details. And the immersive cameras wide field of view also affected how we lit the film. So we built as much lighting into the set as possible. Not only did this help with the creative vision so we could maintain the authenticity of the sub design, but for us producers, it also meant that we could save some costs on the back end so we wouldn't have to do as much visual, visual effects cleanup, because anything that sits inside of your 180 degree field of view that isn't part of the set will need to be removed in post, and we learned that can get extremely costly very quickly because of our high frame rate and resolution. And while we filmed most of our scenes inside a stage, we did have to go on location for the final open water scene.

So that is the immersive camera in a splash housing, and that protects it from the water. And getting to that final shoot day was extremely challenging. We faced a lot of pressure to get that location right. For instance, we had to be extra mindful of the weather because we wanted to keep a stable level horizon to avoid Motion discomfort in the Vision Pro. So waves. They were not our friend, and we also needed a location with a fully unobstructed field of view. So we had to pay extra attention to what else was out in the water. Our teams. They pored over shipping routes to ensure that we wouldn't have a lot of traffic from other vessels. These are just some of the many, many considerations we had to take into account when producing this shoot. There were also many learnings for actors on screen that leads us to our next area impacted. The talent on screen presentation, like our set design because of the acuity of our cameras and when the world scale is 1 to 1. Details Matter elements like wardrobe and makeup and actor performance. They had to be authentic. For instance, some of the makeup didn't hold up to our camera tests. When we transferred our tattoos over to our actors, they didn't feel real enough in immersive, which was not something we were expecting. But okay, so this brings me to a really important. Sorry. This brings me to a really important difference between regular 2D production and an Apple Immersive production. And that's how we're reviewing footage. So if you saw the original tattoos on a regular 2D monitor, You wouldn't think anything of it, but reviewing footage in Apple Vision Pro is the only way to truly assess your shots when you're in production.

Traditionally, the review process on a 2D production is fairly straightforward. After a take, you have your playback operator playback footage directly to your monitor. You watch your dailies the next morning. It's a pretty well oiled machine on an immersive production. While you can watch your playback flat, it doesn't give you the most accurate representation of your footage.

So we discovered it was necessary to have many Vision Pro devices in circulation, and we also needed to ensure everyone knew how to use a Vision Pro, so watching footage wouldn't delay the production day.

We built platforms for everyone's Vision Pro, and like you would with any other device, we had to keep them clean and charged. So this taught us that it was actually crucial to have one person dedicated to keeping the Vision Pro devices running a Vision Pro manager, and this was something we also needed to account for. I know you have to have that though, I promise you. So. This is something we also had to account for in the daily schedule. This whole process, it does take a little bit longer. Which leads me to our last learning, another major technological element that also impacted our production schedule the use of additional camera systems in Submerged. We had a number of shots that were not at a 1 to 1 world scale. They were much closer and they had a shallow depth of field like you see here. These shots were filmed on a completely different camera system. It was a traditional cinematic 3D rig, and we learned that we could not seamlessly integrate this camera system into our production workflow. It had a separate team dailies process and other implications in post-production. So this is another reason why. Early camera testing Attesting early camera testing is so important, so you can understand the needs of any additional camera units and how they'll integrate into your immersive production. So if there is anything I can leave you with when it comes to new technology and production expectations, spend enough time preparing yourselves. Be open minded while learning how to integrate this technology into your productions. And lastly, get excited. Okay, how often is it that we essentially get to go back to film school and learn something so new? And through all of our collective experiences, we can help each other produce successful immersive projects.

And now let's step out. Step out of the scripted world and into a thrilling live arena. It is my pleasure to introduce Ivan, who is going to walk you through some lessons in accessing and capturing live events. Thank you.