Tailor the VoiceOver experience in your data-rich apps

Hello, I am Nandini Sundara Raman, and today, I will be showing you how you can tailor the VoiceOver experience in your data-rich apps using AXCustomContent API. VoiceOver is Apple's screen reader that lets you interact with any Apple device, even if you can't see the screen. You can use VoiceOver by touching the screen to listen to what is under your finger and then perform simple gestures like swipe to navigate the interface. So, here I have a simple app that I created called Woof Woof Kennel where we have detailed descriptions of several dogs listed in a table view, and VoiceOver user can swipe through the list to explore the kennel. Let's have a quick look of the VoiceOver experience of this app. Bailey, beagle, happy-go-lucky, excellent hunting dog, and loyal companion. Popularity, number one. Age, three years. Weight, 25 pounds. Height, 14 inches.

Winston, pug, loving, intelligent, affectionate, loyal, charming, playful, and mischievous. Popularity, number two. Age, three years. Weight, 14 pounds. Height, 10 inches. Whoa! That was a lot of data read by VoiceOver for each dog where all the contents of the cell was read from top left to bottom right. While navigating an interface with VoiceOver, too much data can be overwhelming, with cognitive load on users, and make your app hard to navigate.

And too little data will result in incomplete experience of your app. Both extremes are frustrating, and finding a balance here might be difficult because different users might want different information when using your app. To achieve this balance, we have the Accessibility Custom Content API. This class and protocol in Accessibility framework is available on all platforms.

A common issue with presenting complex data with Voiceover is solved by this API by delivering accessibility information to your users in measured portions.

With this API, you can leverage assistive technologies to present only the content your app users need when they need it. Now, let's look at how AXCustomContent API can be implemented in our data-rich apps. And to do that, first I would like to show you how my app looks with AXCustomContent API implemented. Bailey, beagle. More content available. Yes! You heard that right. VoiceOver just announced that we have more content available on this cell. By this, it means that you can access additional content on this cell using the More Content rotor. Voiceover rotors enable users to navigate applications quickly and efficiently. It has many built-in uses such as setting the VoiceOver language, adjusting the speaking rate, and also moving between the links on a web page, etc. Here, with More Content rotor, we can navigate between additional content of an element under the VoiceOver cursor. To get to the More Content rotor with VoiceOver in iOS devices, twist two fingers on the screen as if rotating a dial till you reach the desired rotor option that is the More Content rotor. More content. And then, swipe down or up to navigate to the next or previous additional content available under the VoiceOver cursor. Now, as I swipe down the cell, VoiceOver will present additional content present on it. Three years, age. Number one, popularity. 25 pounds, weight. 14 inches, height. Happy-go-lucky, excellent hunting dog, and loyal companion. Description. Let's see how the macOS version of the same app works after implementing AXCustomContent API on it. Bailey, beagle. More content available. Once the VoiceOver cursor is on the element with more content, we can bring up the rotor with VO + command + / to reveal its custom content.

More Content menu. Five items. Age, three years. Popularity, number one. Weight, 25 pounds. Height, 14 inches. Description, happy-go-lucky, excellent hunting dog, and loyal companion.

In iOS, to get notified when VoiceOver element has additional content, go to VoiceOver preferences and open Verbosity settings.

Under Verbosity settings, select More Content.

And choose either to speak hints, play sound, or change pitch to get notified when additional content is available. Similarly, you can set the verbosity of more content available announcement in macOS as well. You can do this using VoiceOver Utility preferences. Now, let's see how we can make our Woof Woof Kennel app support Accessibility Custom Content API.

Here, we have the DogTableViewCell with variables for dog photo, name, type, description, popularity, age, weight, and height.

To implement AXCustomContent API, we first import Accessibility framework.

Then we need to add and implement AXCustomContentProvider Protocol on our TableViewCell.

Before implementing the protocol methods, let's override accessibilityLabel property and return just the name and the type of the dog, as they are the most relevant details here.

We then implement AXCustomContentProvider protocol method accessibilityCustomContent with an array of AXCustomContent objects where AXCustomContent objects are a key-value pair of localized strings.

Here we create AXCustomContent object for description with string "Description" as the label and description's text value as the value of the AXCustomContent object. We then also create AXCustomContent objects for popularity, age, weight, and height as well. We then return an array of all these AXCustomContent objects in the order you want VoiceOver to present them to your user. Now, let's see how the VoiceOver experience has improved after implementing AXCustomContentProvider protocol on our TableViewCell. Bailey, beagle. More content available. Number one, popularity. Three years, age. 25 pounds, weight. 14 inches, height. Happy-go-lucky, excellent hunting dog, and loyal companion, description. For a user who is trying to browse this list, the age of the dog might be a crucial information for them if they want to adopt it. Currently, its value is hidden behind the More Content rotor. AXCustomContent object has a property named "importance" that can be used to solve this problem. We can set this property's value to "high," and the content will always be presented when VoiceOver focuses on the element without cycling through the additional elements. To ensure that the age of the dog is always presented by VoiceOver, we have set the importance property on the age custom content object to be "high." Now, here we see that the age of the dog is always read by VoiceOver. Bailey, beagle, three years. More content available. This age information will also be available in your More Content rotor. New this year is that we support the same capability with SwiftUI as well. We can set the accessibilityCustomContent modifier on our custom views in SwiftUI by passing in localizable text for both label and the value parameters and set the importance property as needed. This way works well in situations where the custom content value is added in one place and not manipulated later in the hierarchy. To implement this in our DogCell View, we first set the accessibility hidden modifiers on description text and on the HStack that contains popularity, age, weight, and height of the dog so that these aren't automatically picked up for the accessibility label of the DogCell. We then set accessibilityCustomContent modifiers for age with importance set to high, popularity, weight, height, and description. If we want our custom content value to be manipulated in more than one place in the Swift UI hierarchy, we can avoid redefinition of the key by creating an extension of AccessibilityCustomContentKey. This will give the ability to update or remove the value for that key without recreating the localizable text string. In our DogCell example, we have created an extension of AccessibilityCustomContentKey and added age as one of the custom keys. On the DogCell body's VStack, we have added accessibility CustomContent modifier with the AccessibilityCustomContentKey "age," instead of using a string key here. Finally, here is what we hope you will do to tailor the VoiceOver experience of our data-rich apps. First, identify visually data-rich parts of your app. Turn on VoiceOver and determine if the content under the VoiceOver cursor is too verbose. If it is verbose, implement AXCustomContentProvider protocol and consider moving the supplementary information to accessibilityCustomContent. Set the importance property to "high" for the information you always want to present to your user. For the VoiceOver users, to have an option to access their content in measured portions and have the content presented to them only when they need it will greatly improve the Accessibility experience of your app. So now, to give a better experience to our VoiceOver users, I really encourage you to take a look at how you can incorporate AXCustomContent APIs into your data-rich apps! And thank you so much for watching this session, everyone. Bye-bye! [percussive music]