Shop online with AR Quick Look

Hello, and welcome to WWDC.

Hi, everyone. Thank you for tuning in.

My name is David Lui, and I'm excited to present to you how to bring AR Quick Look to commerce and retail workflows.

Starting in iOS 12, we introduced AR Quick Look, our built-in system-wide AR viewer for 3D content where you can preview USDZ and Reality File scenes right with you in your world, all without needing to download an app. With iOS 13, we added new features like launching straight into AR the moment you tap the 3D model from any application or website. You can now share product links instead of the 3D model to bring users back to detailed product pages from retailers.

We added support for detecting vertical planes along with horizontal planes making it possible to anchor picture frames or lights on floors and walls in your space.

There's even support for viewing face accessories, like glasses and masks, using the front camera coupled with our automatic face occlusion for that realistic try-on experience.

We gave a session last year where you can learn more about our other features and advancements in AR Quick Look. In the last year, AR Quick Look was accompanied by two new products in the AR family...

Reality Composer and Reality Converter, and serves as a great visualization tool for viewing and sharing your 3D compositions.

And in support of those applications and frameworks, like ARKit and RealityKit, we've added new features like instant AR placement and object occlusion that make the AR experience more responsive and realistic, especially on the new iPad Pro with LiDAR scanner.

Now objects appear immediately in your world and can be anchored anywhere from the flat surfaces of your floor to the curved surface of your couch. And together with object occlusion, AR furniture could be placed in your space between and behind pieces of real furniture for more realistic product visualization. But one of the amazing things we previewed last year was our retail integration, and we started with Apple Pay. Our goal is to provide a tighter integration of AR technology and commerce with retailers in a really simple and easy way, and AR Quick Look was a fantastic solution to enable not only a richer and more immersive experience for consumers, but also for businesses to sell their products in AR. We've seen a lot of great companies like Home Depot, 1800Flowers, and Wayfair adopt this technology and integrate AR Quick Look as part of their shopping experience with great feedback, resulting in increased purchases. And one of the most notable is Bang & Olufsen, where Jakob Kristoffersen, Senior Manager of Customer Experience said, "Users who engage with the AR experience in our iOS app or on the website are four times more likely to seek out our Store Finder to visit a B&O store and has aided our sales effort to great extents." So let's take a look at how this works. We've added a banner at the bottom of the AR Quick Look view which provides a quick one-tap way for the customer to immediately purchase the item, like this alarm clock, after seeing the product in the world with AR.

The Apple Pay banner has a very clean and templated layout where the product information is located on the left and the Apple Pay button is on the right. In addition to the Apple Pay banner, there is a custom action banner where retailers can provide their own call to action text.

Our third banner style is entirely customizable, so you can completely configure the look and feel of the banner using custom HTML. These three banner styles are available through the web starting with iOS 13.3 on iPhone, iPad, and iPod touch.

Let's take a look how this experience works from end-to-end and discuss what's needed to create the banner.

Suppose I run an online store selling retro alarm clocks and I want to add the Apple Pay button to my AR experience when customers visit each product page. On my web page, I can add a link to my USDZ model so that customers can preview the alarm clock on their nightstand using AR Quick Look.

Last year, we introduced a way to customize the AR Quick Look experience through the web URL's fragment identifier. We can use the same system to provide the required information for AR Quick Look to use to display an Apple Pay banner.

These customization options get passed to AR Quick Look when the user taps on the link. AR Quick Look then parses the fragment identifier and creates the appropriate banner style.

The last step for me is to add an EventListener to my page. This way, when the customer is ready to take action, tapping on the banner will dismiss AR Quick Look, and notify the EventListener on my website, which can then present the Apple Pay prompt. Now, let's go into more detail about how to specify these customizations. Before we jump in, I'd like to first give you a refresher on how to integrate AR Quick Look with your website.

On my product page I've added an "a" tag with a link to the USDZ model of my retro alarm clock. I've also nested an image tag to show a thumbnail of the clock.

Adding the rel="ar" attribute tells Safari that this is an AR experience so it will draw an AR badge and will present AR Quick Look directly, instead of first navigating to another page.

Next, we customize the experience by passing in customization options through the web URL's fragment identifier, which appears after the hash mark. To help with code readability, I've separated each parameter on its own line, but in practice they should be separated only by an ampersand without any spaces in between.

Normally, tapping on the share button allows the user to share the current page that they are on. However, specifying the canonicalWebPageURL overrides the default URL. In my example, users can share a direct link to this fictitious product page, even though they were originally viewing it on the Quick Look gallery page.

We also support disabling content scaling through the allowsContentScaling parameter, so people can preview content at the intended size. In my example, I always want the alarm clock viewed at its true real-world size, so I've provided a value of zero to allowsContentScaling which disables scaling. So that's a quick recap on integrating AR Quick Look on your website and how to use the fragment identifier to customize AR Quick Look. Now let's talk in more detail about each of the three available banner styles...

starting with Apple Pay. If you're already accepting Apple Pay as part of your business payment methods, the Apple Pay banner is the simplest and most convenient way to get set up and allow customers to make purchases right within the AR experience with all the benefits of Apple Pay, like being a fast and secure way to pay. If this is your first time incorporating Apple Pay on your website, there's a lot of great resources online including guidelines to help get set up.

As a reminder, AR Quick Look is a system extension and was built with security in mind. So no payment information is collected or stored since the payment happens after the AR experience dismisses and the customer is returned back to the retail web page.

Retailers have a wide selection of button types to choose from to best match with their business. Specifically seven types. The first four button types, like Pay with or Check out with Apple Pay, are focused on purchasing a product... while the last three payment styles, book, donate and subscribe are tailored for specific use cases, such as travel or subscription-based businesses. Underneath each button type is their corresponding value for the Apple-Pay-Button-Type parameter option, which we'll talk about shortly.

Here's the code sample which will present AR Quick Look with the Apple Pay button. The first thing you might notice is there's an additional id="ar-link" attribute. This specifies the unique identifier for this link element which we will use later when we start listening for tap events.

There are four required parameters: applePayButtonType, checkoutTitle, checkoutSubtitle and price.

In our example, I've chosen the plain Apple Pay style, which is simple and compact.

The title and subtitle is where you can supply the product information to the customer.

Because URLs can't contain spaces, the values in the fragment identifier need to be percent-encoded such as the spaces in the title and subtitle.

If your website supports multiple languages, you'll need to localize the string values provided to the parameters since AR Quick Look will display them as is. This is especially important for the price parameter when representing currency for different locales.

And of course, you can also combine this with other AR Quick Look customization options like allowsContentScaling.

Let me show you what the full Apple Pay experience looks like. Say I'm shopping and I find an alarm clock that I like. I can place it on my nightstand and see it up close, right in my room.

Oh, this looks great. I'll rotate and position it to my preferred spot close to my bed.

Okay, I'm definitely going to purchase this. Since the banner's always visible on-screen with the Apple Pay button, I can immediately tap "buy" to start the transaction. AR Quick Look will then dismiss and return to the web page with the Apple Pay prompt brought up, where I can then authenticate and complete my purchase. Now let's look at the second style-- the custom action banner. The custom action banner provides the same intuitive layout of the Apply Pay banner but provides the ability for supplying your own call to actions like "Add To Cart" or "Preorder" to help move the customer one step further in the purchasing flow.

This conveys the intention by the user to proceed to follow-up questions such as asking for an e-mail address to be notified when the kids slide is released.

After AR Quick Look dismisses, your website can then perform custom logic such as adding to the item to the shopping cart, prompting for the product size, or even opening Apple Maps to help the customer find the closest store near them.

The call to action provides more flexibility to enable additional retail use cases besides purchasing, providing more ways to integrate with AR.

The code is very similar to Apple Pay except we need to specify callToAction instead of the Apple-Pay-Button-Type parameter. Again, be sure to percent-encode the URL parameter values in the fragment identifier and localize the strings for different locales for both the call to action text and price fields.

In addition, the price parameter is now optional for the custom action in iOS 14. This provides better flexibility for products which might have varying prices depending on the product variant. However, keep in mind that this field is still required on previous versions of iOS and should not be empty. Let's take a look at a kids slide from the AR Quick Look gallery page. I'll go into object mode, where you can see the custom action banner is always visible for quick access. Tapping on the custom action button will dismiss AR Quick Look and take me to the documentation page, which is how I've defined and set up the custom action.

Now let's look at the last style, the custom banner. This banner is the most customizable, where you can supply your own custom HTML file to fit your own unique use case.

This allows you to customize the entire view so you can use your own font, layout and graphic to match your business' branding and style.

This banner supports three predefined heights which can be used to communicate additional information and contain personalized messaging.

Like the custom action, it is up to you to decide how to respond. For example, you can start a Business Chat conversation by redirecting to your unique Business Chat URL.

To comply with AR Quick Look's security standards, the HTML resource must be sent over HTTPS. Otherwise, the banner will not display and the request will be ignored.

To display the custom HTML banner, you'll need to set just one parameter, "custom," and provide the absolute URL to the HTML file. Like the other two banners, make sure to percent-encode this value before appending it as a URL parameter.

There's an optional "customHeight" parameter available if you want to set the banner height. Supply a value of 'small' to set the banner height to 81 points. Or 'medium' to set the banner height to 121 points. And 'large' to set the banner height to 161 points. If you omit the custom height URL parameter, AR Quick Look uses a default small height.

Here's a great example of shopping for a guitar from our fictitious store, called Acoustic House. I'll go ahead and tap on this guitar image which will push me into AR Quick Look where I can preview the Stratocaster guitar in my room.

You can see Acoustic House is using the custom banner style and has presented the banner for a conversational experience with the immersion in AR. On tap, AR Quick Look launches a Business Chat conversation in Messages, where I can chat with a live agent and ask questions and eventually place an order. This illustrates how AR Quick Look provides a really powerful handoff from the AR experience to applications like Business Chat, allowing customers to connect directly with companies to learn more and quickly get answers.

If you're not already familiar, Business Chat allows consumers to connect with brands to ask questions, schedule appointments and deliveries, purchase goods and services using Apple Pay and more right from the Messages App. Here, in this example, The Home Depot is using AR Quick Look to help consumers check out home appliances in an immersive and engaging way as part of their buying process.

And with our new banner feature, AR Quick Look can then launch Business Chat where consumers can start a conversation with businesses like Home Depot about the washer machine and receive a USDZ file of the companion dryer where they can place it virtually in their laundry room with AR.

This is a great way for customers to learn more about a product and really make an informed buying decision in a private and secure channel. And for businesses, it is really easy to provide this seamless customer experience by using a Business Chat URL as a custom action or custom banner event.

Here's a summary of the different customization parameters for each of the different banner styles. As a reminder, the highlighted parameters are optional for that banner style.

Now that we've gone over how to customize AR Quick Look to show a banner, let's go over how your website can get notified when users interact with the banner.

From earlier, we saw the code snippets of how to use the fragment identifier to specify which type of banner to show. To listen for and be notified of tap events, we need to add an EventListener like this.

Remember the id="ar-link" earlier? This is how the script is able to retrieve the corresponding "a" tag that we defined previously and assign it to the constant linkElement.

When WebKit invokes your listener, check the data property. A value of "apple_ar_quicklook_button_tapped" confirms the user tapped the banner in AR Quick Look.

From here, you can either invoke the Apple Pay sheet, or perform some custom logic like adding the item to the shopping cart depending on the banner style displayed.

The AR Quick Look gallery has been updated with new 3D models and the banner examples we've gone through together. Visit the gallery page on your iOS devices and try everything out.

It's really amazing to see how people are taking advantage of AR and integrating AR Quick Look into their website and iOS application workflows. We love seeing all the many types of content people are sharing and experiencing in AR Quick Look. You saw the Apple Pay and Custom Action banner which are really simple and easy templated styles to quickly add purchasing options to your AR experience. With the custom HTML banner, retailers have even greater flexibility in customizing the view to match their branding and colors.

There's lots of great documentation available on developer.apple.com, including how to add an Apple Pay button or a custom action in AR Quick Look that I've went through today. We love to hear from you, and you can send us feedback on your Apple Pay and custom banner integrations, along with any requests you may have, through Feedback Assistant. Thank you, and thanks for watching.