Meet Safari for spatial computing

♪ Mellow instrumental hip-hop ♪ ♪ Etienne Segonzac: Hi there! Welcome to our session. I'm Etienne Segonzac, an engineer on the Safari team. And I'm joined by my teammate, Tim Horton. We would like to introduce you to the new Safari for spatial computing. It's not every year that we get to bring the web to a brand new platform, so this session means a lot to us. We took great care to build the best browser possible. And the web really shines on this new device. So let's start by taking a look at Safari in action. Tim Horton: Thanks, Etienne! Here we go! When you first open Safari on xrOS, you'll notice it looks familiar, just like the Safari you're used to from iPad and Mac. The similarities are more than skin deep. This truly is Safari with the same WebKit engine underneath, plus some thoughtful additions for this platform. All of your websites will work beautifully out of the box, thanks to its extensive support for web standards. Browsing the web is easy. To follow a link, you can reach out and touch the page, or simply look at the link and tap your fingers together. While you're looking at a link, Safari will give it a gentle highlight, helping you to confidently navigate. As you're using Safari, you will quickly find ways in which it makes full use of the power of this platform. For example, tab overview has been completely redesigned for the expansive display, and makes switching tabs more enjoyable than ever.

The platform's support for multiple windows lets you multitask like never before, surrounding yourself with websites and other apps, arranged however you'd like. I particularly like using this with Mac Virtual Display for the ultimate web development experience. And bringing your favorite web videos into full screen will really bring them into focus. ♪ Mellow instrumental hip-hop ♪ Hi there. I'm Kendall Bagley, a software engineer on the Safari team. While we've brought the web as you know it to this platform, there are still best practices to keep in mind, so let's go back to Etienne to hear about some of them. Etienne: Thanks Tim. I just love tab overview! And what's amazing is that all of these websites work out of the box. Thanks to responsive design, they can adapt, from being in the palm of your hand on an iPhone, to filling an entire room. The techniques you already use to progressively enhance your website still apply on this platform. You should design your layouts with CSS viewport units and react to window resize with media and container queries. For graphics, prefer SVG, especially for UI elements. This will ensure the best rendering possible, even when the window is up close. And when you have to use bitmap assets, devicePixelRatio and responsive images will reflect the recommended resolution for image loading and canvas rendering. Today, we will learn about the natural interactions that are core to this new experience, optimize our website for this platform, and get excited as we look into unique opportunities for 3D content on the web. Let's dig a little bit deeper into direct and indirect gestures. Our main input model on this device is driven by the fusion of eye and hand gestures. As you saw during Tim's demo, it's very natural. You simply look around, tap your fingers together, or pinch. When the pinch begins, your eyes will be used to find the HTML target and position the pointerdown event. During the gesture, pointermove events will help you follow the hand motion, and a pointerup will be dispatched on release. You can also reach with your index finger and directly touch the page. When your finger intersects with the window, a pointerdown event is dispatched based on your hand position alone. Pointermove events will track motion as they did before, and the pointerup will be dispatched when your finger stops intersecting with the window. Of course, you don't need to worry about low-level pointer events for simple interactions. Safari still triggers click events, and scroll and scroll snapping work as expected. When it comes to media queries, this primary input model is similar to a touchscreen. The pointer is coarse and doesn't support hovering. But keep in mind that a trackpad or a keyboard could be connected via Bluetooth. You might be wondering why hover styles aren't triggered while you look around the page. That's because this device comes with a new way to provide visual feedback. When a user looks at an element, it will highlight before they tap on it. So neither your website, nor Safari itself, needs to know where the user is looking. This system is based on what we call Interactive regions. They are automatically generated by Safari's WebKit engine based on your accessible markup and CSS styling. Buttons, links, and menus, or elements with the corresponding ARIA roles will automatically get highlighted. Same goes for input fields and form elements, even when they have custom styles. But make sure to use cursor: pointer; for other HTML elements if you want to indicate that something is interactive. Let's see how it works! Here, I'm building a file listing UI, and I have a custom element for items in the list. I'll add cursor: pointer to make them highlightable. But this custom element is using divs internally, and the cursor property is inherited, so the icon and label will get their own region and highlight separately. Setting pointer-events: none on these inner divs will help me clean things up. WebKit will know that they don't need to be individually highlighted. And it will also simplify my event handling code, as I will always get the same event target. I can also shape the highlight by using the CSS border-radius property, adding a radius to every corner of an element, or only some corners. I can even make a perfect circle, but it's important to match the visual style of each element. Hey Tim, remember the website we were working on? I wonder if we need to refine the interactive regions on it. Tim: Yeah, let's see how they look right now! You can use the xrOS simulator to debug highlight regions, even if you don't have a device. Moving the mouse cursor around the window simulates where you're looking, and clicking simulates a tap. Let's open Safari in the simulator and see what we can do to beautify the highlights on my website. The first thing that I notice when looking at this page is that the buttons in the navigation bar don't highlight at all! You may recall that Etienne mentioned that Safari uses the cursor CSS property to determine if an element should be considered interactive. I happen to know that these buttons also don't get a hand cursor in macOS Safari, so I have a hunch that that might be the problem, but we could also check in Web Inspector. If you want to learn more about that, you can watch the "Rediscover Safari Developer Features" session. Using the Web Inspector, I found this erroneous global cursor style. Let's remove it, and let links be links with their default cursor: pointer style.

Hey, great! Now we get a highlight, but it seems a little misplaced. In reality, it's revealing a mistake in my website. Even on macOS, only the text is actually interactive. If I try to click the button itself, it won't follow the link. So we've found a mistake in the website and by fixing it, we can also improve its appearance on this platform! Seems like an all-around win to me. I'll make the whole button a part of the link, instead of only the text.

Let's see if that worked.

It looks much better to me! Now the whole button is interactive, and the highlight covers the whole area. If you look very closely, you'll notice that the corners don't quite line up, though. Safari's highlights have a small radius by default, but if your interactive element has a border-radius, we'll take that into account. Let's fix it up.

Just add the same border-radius to the interactive element.

Perfect! Now the buttons highlight just as we want. Let's tap on one of them.

Oh, those scores animated kind of fast compared to iPad. I guess we have more things to fix, but we'll come back to that one later. For now, let's go back to Etienne to hear about some optimizations you can make to your websites. Etienne: Thanks, Tim! We do have to keep in mind a few things when optimizing for this platform. See, when we introduced Retina displays back in the day, the concept of a pixel became a little bit more abstract. As web developers and designers, we work with CSS pixels and let the user agent adapt to the hardware by using as many device pixels as necessary. Today, the concept of a screen is becoming a little bit more abstract. When you request to go full screen, your goal is to focus on a single element. The rest of the page fades away, and at the same time, the window is resized to a default size. This default size is also reported back to JavaScript as the screen dimensions, so websites expecting the window and screen size to match while in full screen will continue to work. But remember full-screen windows can be resized on this platform. And they can even get larger than the reported screen dimensions. When it comes to scrolling and animations, as always, performance is key, especially on this device, where every animation will target the best frame rate possible. Passive scroll event listeners will prevent your animations from interfering with scrolling. And when using requestAnimationFrame, always measure the elapsed time between each frame. The Web Inspector Timelines will also help you identify any performance issue. You can learn more about them in the Web Inspector tech talk.

Speaking of requestAnimationFrame, my animation here is a bit jumpy. Its runtime depends on the frequency at which we're calling animate(). And this frequency could be higher or lower depending on the device. What I should do instead is measure the time between each update and use this to compute the animation's progress. It's simple enough and it will make my animation independent from the requestAnimationFrame refresh rate. I think that was the issue with the score animation! Tim: Oh yeah, I've already fixed it up. Let's see if it works on device.

That looks much better, and now it's future-proof against devices with any frame rate, slow or fast. The teaware all looks pretty nice on the website, but we can go further. If you want to see how they'll really look on your table, it's very simple to adjust your website to take advantage of Quick Look and make that a reality. Exactly like for AR Quick Look on iOS, you simply add an anchor referencing a USDZ file around an image, and it turns a simple image into a magical Quick Look experience. Amazing! Now I can really tell which one fits in. Etienne, which teaware would you pick? Etienne: I think I'll have to try it out in my kitchen to make sure I find the best match. But this, this is impressive. The demo was really great; it was fast and fun to navigate. And Quick Look really brought things into a whole other dimension. Let's see how you can integrate 3D content on your website. AR Quick Look was originally introduced back in iOS 12. All you need is a link pointing to a USDZ file with a special attribute and an image tag that you can use to show a preview of the 3D model. This exact setup will work on xrOS, providing your users with an easy way to put 3D objects in their space, all while benefiting from the advanced lighting and rendering of RealityKit. Make sure to watch "Explore the USD ecosystem" if you want to learn more. Along the same lines, the HTML model element is being proposed as a potential W3C standard. Think of it as AR Quick Look inside of the page, or like an image tag for 3D objects. It will yield the best rendering possible on every device, all the way to full stereoscopic view and environmental lighting. Introducing an entirely new element is a big step, so we're starting simple with an easy way to specify the source of your 3D object and an attribute to enable or disable user interactions. The JavaScript API is a bit richer. You have access to the camera, animations, and more. Sounds interesting? If you run the latest Safari, you can enable the model element feature flag on any platform. And If you want to take things even further, check out the developer preview of WebXR, a W3C standard to build fully immersive scenes on the web. WebXR is based on WebGL, and many popular WebGL libraries already have built-in support. If you have the code for a 3D scene, you can make it fully immersive by requesting a WebXR session. It's nothing like panning around with a mouse or a trackpad. You will quite literally be transported inside the scene. You have to try it to believe it. On xrOS, you can find the WebXR feature flags in the advanced settings for Safari. And this is just the beginning. There's so much more to explore when it comes to the immersive web. Technologies like CSS transforms and pointer events are bound to evolve on this new platform. This is why Apple is actively involved in the definition and development of W3C web standards. And you have an opportunity to provide input. Please let us know what you think and what you're looking forward to. You can start experimenting with new 3D experiences today. And interactive regions will be easy to review on the xrOS simulator. If you find any bugs, the Feedback Assistant is available on all platforms. And for anything related to HTML, JavaScript, or CSS, the WebKit bug tracker is at bugs.webkit.org. Speaking of CSS, there are many new features coming to Safari this year, so don't miss "What's new in CSS." Thank you for joining, everyone! Tim: And we'll see you on the immersive web! ♪