What’s new in USD and MaterialX

Hi, I’m Lori Smallwood, an Engineering Manager for 3D content creation workflows in the Vision Products Group.

In this talk, I’ll go over how we’ve made the creation of 3D content easier than ever before when using USD and MaterialX. I’ll outline how these technologies provide a strong foundation for 3D content creation and interchange.

Then, I’ll tell you about some USD and MaterialX updates in RealityKit followed by advancements in creative tooling on macOS.

With USD and MaterialX, Apple has embraced the use of industry standard, open source software to form a strong foundation for creators like you to develop content. These technologies work together to streamline content development across a range of applications and industries. OpenUSD, originally developed at Pixar, enables the standardized interchange of 3D assets and scenes.

MaterialX, which was developed at Industrial Light and Magic, is used to portably represent shaders which provide the visual look of the 3D content.

USD and MaterialX empower you to capture your creativity and bring it to life in front of your eyes. From immersive feature films, to indie games and beautiful spatial experiences, you can use the same workflows between all your projects, and in the applications in which you’re most comfortable. Vision Pro really highlights the scalability enabled by consistent workflows through apps like Apple TV+, Max and Disney+, which let you be immersed in environments created with the same technologies powering the films you watch within them. The images you see on your displays are produced by rendering engines. To streamline the creation of spatial experiences, we’ve made several advancements in two of the rendering engines available on our platforms: RealityKit and Storm. Both support USD and MaterialX, and cater to different production needs. RealityKit is a powerful realtime engine that can help you create amazing spatial experiences.

It powers visionOS, and applications like QuickLook on your phone and Reality Composer Pro on your Mac. You can use RealityKit in your own applications to leverage the unique capabilities of our platforms.

Storm is Pixar’s real-time renderer included with OpenUSD. It is commonly used in content creation applications like DaVinci Resolve, and on macOS we make it available in Preview to provide a familiar and convenient experience for professionals targeting those workflows.

RealityKit has some great USD and MaterialX changes in store this year. Last year we announced Vision Pro and visionOS, where we introduced a number of new RealityKit features to enable gorgeous content. This includes support for MaterialX shaders in ShaderGraph, a shader creation system within Reality Composer Pro.

Now we’re bringing those same features to our other platforms, so you get the same great visuals across visionOS, iOS, iPadOS and macOS when using RealityKit.

Shadergraph is a great way to design shaders. It makes experimentation fun, fast and all without any coding.

We have several bespoke nodes that let creators make the most of our platforms. We’re making many of these available online to integrate into the content creation applications you use, to streamline your workflows. These nodes may function a little differently across apps, since those apps have their own rendering choices and capabilities. The node definitions and an intro to each node are available in our Shadergraph documentation, linked in the session description.

We have some USD feature updates to share with you as well. We’ve added support for blend shapes and subdivision surfaces to USD in RealityKit, which can enable you to create expressive characters, similar to Animoji. Blend Shapes allow your characters to emote with designed smiles, frowns and anything in between.

Subdivision surfaces allow you to have incredibly smooth objects at runtime, while avoiding high polygon counts in your assets. More information about these features, and their performance considerations, can be found in our developer documentation linked in the session notes. In USD it is possible to support multiple representations of an object, such as a color or shape, using a feature called Variants. Here you can see how you can choose between different planet variants.

QuickLook now supports the ability to switch between variants interactively. Note that variants must be declared on the default prim in your USD scene.

The RealityKit API also supports the ability to specify variants when loading a USD file. Check out Naveen’s talk, “What's new in QuickLook for visionOS” to learn more.

Macs have always been a great platform for creators, and we continue to extend that for 3D with USD. This year we’re adding more tools to your toolbox to make it easier for you to work with 3D content and give you more time to create the things you enjoy. Preview is the simple yet powerful system-provided viewer and editor for various media types on macOS, including USD.

We’ve made it much easier to check and correct the size and orientations of your 3D models in Preview. Want to scale down some assets so they fit inside your digital dollhouse? With the new Adjust Size tool in Preview, you can make those kind of adjustments without launching or even learning a 3D editor.

You might want to reduce your file size, for example, when sharing assets online to make them faster to download. Preview also gives you the option to export with smaller, compressed textures.

Lighting is important for evaluating your assets, and we’ve updated the lighting used by the Storm renderer to more closely match RealityKit.

Storm also now supports MaterialX, including many Shadergraph nodes. The lighting and MaterialX updates to Storm provide a more consistent look and feel to your assets across our platforms and applications. Do note that because the renderers inherently target different use cases, there may still be differences in how images render, and which nodes are supported. Finder is where you navigate project files, and it’s seen improvements as well.

Converting files to USDZ is just a right-click away, with the new Convert to USDZ action available in the Shortcuts app.

And Archive Utility now extracts USDZ for you, also in the right-click menu.

Finder can now load pre-rendered asset preview thumbnails. This is useful if you want to quickly browse through directories with large files which might take a long time to render and for files where you want to have a custom preview thumbnail. This makes browsing through Finder faster.

In general, Finder better manages folders and renders previews of large USD scenes without affecting system performance. We’ve also upgraded our lower level system libraries and tools, providing more functionality for technical creators.

Creators in non-English locations can now use Unicode names for their prims, thanks to upgrades in OpenUSD by NVIDIA and Pixar. For example, Hindi and Chinese names can be used, allowing USD content to be accessible in more regions than before.

We’ve also included several command line tools in macOS for technical creators who want to inspect their USD files or to automate workflows at scale.

Some of these are familiar if you use USD and others are new. For example, usdcat allows conversion between formats, usdchecker allows you to validate your files, usdzip lets you create portable USDZ packages, and usdcrush provides tools to compress USDZ files.

At Apple, we believe that building upon powerful technologies like USD and MaterialX is fundamental to creating great 3D content.

The motion picture industry also shares our appreciation of OpenUSD, and we’d like to congratulate Pixar, and the OpenUSD team, for receiving an Academy Award for Scientific and Technical Achievement. This further underscores that USD is a foundational technology for 3D content generation.

We’re also proud to have founded the Alliance for OpenUSD with our partners at Pixar, Adobe, Autodesk, and NVIDIA.

The Alliance for OpenUSD allows us to combine our development efforts and make sure that USD can serve a diverse set of use cases. Since its start in August last year, the Alliance has grown fivefold and includes industry leaders in a range of disciplines like gaming, e-commerce and web technologies. Additionally, the AOUSD are well underway on creating a formal specification for the core of USD, ensuring it’s a technology that can persist into the future.

Check out aousd.org to learn more and engage in our free access forum.

We believe strongly in advancing the industry as a whole, beyond our own platforms, by contributing to the open source ecosystem. Our teams continue to contribute significantly to many projects like OpenUSD, MaterialX, Blender and Maya’s USD plugin. We recommend staying up to date with these, as you may see new features and improved compatibility and performance over time. USD and MaterialX give you the foundation to create anything from single developer spatial experiences to blockbuster feature films. To help you get started with using them in your workflows, we recommend checking out these talks.

We’re excited to empower your creative work, and can’t wait to see what you make.