Design for intelligence: Make friends with "The System"

Hello and welcome to WWDC.

Hey. My name is Sofiane, and I work on Proactive Intelligence here at Apple. Today, I want to talk to you about how your app can make friends with the system. As Radhika showed in the previous video, system intelligence can help people at every step of their journey in using their apps and the devices that those apps are installed on.

The better job you, as a developer, do to integrate with the system, and the more people use your apps, the more we can help them over time. But how do you do a good job of integrating with the system? I'll explain by talking about a few key concepts shared across all of our intelligence technologies.

Then I'll have my colleagues, Mert and Chad, talk about how their teams use these foundational concepts in slightly different ways, all to achieve the same goal: making everyday things easier.

And then Mark will come back to recap exactly what we hope you do next.

So, let's talk about what makes the intelligent system work.

We all live busy lives, and for many of us, our iOS devices have become an important part of how we get things done. People rely on apps dozens of times every day, for tasks ranging from communication, ordering food, or even tracking water intake. And again, our goal is to make these everyday interactions as smooth and frictionless as possible so you can get time back to focus on your life.

And for this to happen, the system and your app need to work together. And by working together, what I really mean is that we need to speak the same language. For that common language, we built the Intents framework. There are three key concepts you should know about, and these are define, learn and execute.

Understanding them will help you accelerate your users along your apps' critical paths, and adapt to their preferences over time. Let's first talk about what we mean by "Define." Ultimately, you, as developers and designers, just need to ask yourself, what are the important actions that people want to perform with your app? The goal is simply to determine what are the key and repeatable things that people want to accomplish. You can define these actions using what we call "intents." Intents represent the "what." What do people do in your app? Ask yourself, "What are those tasks? And what are all of the relevant attributes necessary to carry them out?" What, for example, does it mean to track your water intake? Or what's involved in checking the news? Or what does it mean to order coffee? Let's actually take a closer look at that example.

So, to order coffee, you know you'll need an intent that represents that action. And, of course, people like their coffee made differently. There are obviously different types of coffee. Maybe the size of the drink matters too.

Details like this are the attributes of the action, also known as the parameters of the intent.

The Intents framework gives you the flexibility to define your intent and its parameters, making it easy for you to package all of the information about the action someone is performing inside your app.

Here we're creating an "order coffee" intent and defining two of its parameters, representing the type of coffee item and its size.

With those parameters defined, we can set these values to represent what the user is actually ordering, and here, it's a large iced latte. So, to summarize, the intents you define are a key building block that enable your app and the system to speak the same language.

Now, a key piece of that intelligent system experience is learning how people do things in order to predict how they might do so again in the future. But how do we learn? All it takes is someone using your app, and as long as you, as a developer, do something called a "donation" as people use your app and perform the tasks that are important to them, these donations provide signals that the system can use to learn from. This happens using on-device intelligence, and eventually, we'll detect patterns that help us make predictions, like the Siri Suggestions we surface on the lock screen.

And when someone uses a lock screen suggestion, not only do they save time, but we can tell that we did a good job, and that positive signal helps us make even better predictions in the future. So, let's dive deeper into this concept of donations. Donations help us answer questions like when do I order coffee? Or where does that typically happen? But time and location are only a few of the signals that we use to understand something more important, which is context. As JP said earlier, we want you to feel like you have superpowers. We pull that off when we match the task you want to accomplish with just the right moment. Now, a donation is simply a record, a snapshot in time, of an intent that's actually been executed.

Maybe today, your app donates your large latte order because that's just what you crave as you start the day. And a bit later in the day, you order something more refreshing, like a lemonade.

But the next day, you start your morning off again with another large latte.

And as people continue to use your app, these signals accumulate over time. And while there might be changeups, patterns emerge and the system can learn them. Once we learn these patterns, we make predictions. We try to match the right task to your context, and we get better the more we learn. And again here, privacy is at the very core of what we do. The information you donate never leaves the device. The system uses your donations locally to offer an amazing, personalized intelligent experience. What happens on your device stays on your device. So, we keep returning to this point that an intelligent system should save you time, but it can only do that if we drastically reduce the amount of friction required to get something done.

A key way we do that is by accelerating people to the things they want to accomplish.

So, as a developer, it's important for you to be prepared to perform an action even when your app is not running.

When we make a prediction, we may reconstruct an intent from the information gleaned from your app donations.

When our prediction is right and someone engages with it, we pass the underlying intent back over to you so you can execute it in your app.

Now, your goal is to execute that action, and there are two ways you might go about this.

Whenever possible, you should execute the action in the background. If you support background execution, the action is performed right then and there and the user doesn't have to switch between apps.

You even have the ability to display your beautiful app UI right in this flow. Here, we're showing a dialogue to let the user confirm their order before placing it.

Alternatively, we can launch your app. Here, you should execute the action and take the user right to the relevant part of your app. In that case, it's our order confirmation screen. These were some of the important concepts you should know about so your app can take full advantage of our intelligent system experience.

Now to share more about how teams at Apple use intents and donations in greater detail, I'm going to hand it off to my colleague Mert to talk about Shortcuts. Mert? Thanks, Sofiane.

To illustrate how we use intents and donations, I'm going to use an example some of you might be familiar with called Soup Chef. We've used this as an example app for Shortcuts since we launched, and it's a great way for me to use the real plural of "soup," which is, in fact, "soups." This app is pretty basic. You can launch it, take a look at today's soups and place an order for one of them.

But let's step back for a second. What can you do using the Soup Chef app? You can order one of the available soups, check the remaining delivery time for a soup order, or get your order history. But how does the app actually get these features in front of people? All of these things, this functionality, is contained in the app itself.

So, to build on what Sofiane was talking about, that means I have to stop whatever it is I'm doing at that moment to unlock my device, find the app icon, and maneuver through some layers of that app's UI, all just to get soup. I mean, let's say I'm driving and I really, really want soup. This has been known to happen. I should not reach out to my phone, launch an app, and dig around some nested table views. And, yes, I won't drink my soup until I'm stationary at some later time.

For that, we have Siri. We have CarPlay for the car, HomePod in the home, or even AirPods when you're on the go. There's always some device that can respond to "Hey Siri." So, stepping back into the role of the app user, this is why we often talk about how Shortcuts can accelerate everyday tasks.

Using my voice to ask Siri to order soup is mostly convenient because it makes this thing I already do easier.

And for someone who's looking for ways to make their everyday routines easier, setting up a shortcut and getting into the habit of using your voice is very much worth it. So, whatever that thing is for your app, that killer feature, which may even differ between different people, that's the thing you should expose as a shortcut. And it doesn't need to be limited to a single shortcut either. After placing an order for a delicious soup, I might want to use Siri to check the delivery time until my delicious soup arrives. So, how can you make this a reality? As Sofiane described: intents. For general categories of actions like sending a message, we have system-defined intents that you can adopt. For describing any other action unique to your app, like ordering soup, you can define entirely custom intents in Xcode. When you say, "Hey, Siri, order soup," Siri runs your "order soup" intent.

And when you say, "Hey, Siri, check delivery," Siri runs your "check delivery time" intent.

For more on understanding intents, make sure to check out the Human Interface guidelines. It's a great resource for engineers and designers alike. For your users, adding custom intents to Siri in the form of a shortcut is as easy as tapping on an "Add to Siri" button.

And the parameters that Sofiane was talking about earlier, which describe the important aspects of the thing that's being done even appear right here in the Add to Siri flow.

That's because you can actually change how this shortcut works right when you're setting it up. Let's say I don't actually want tomato soup every time.

I could change it to clam chowder.

Or, if I'm feeling really wild...

I can leave it empty.

And Siri will ask me what soup I want every time I order with my voice. So, once my shortcut has been added to Siri, now this flow can happen without the app needing to be open. Just ask Siri.

Soup time.

I found two restaurants near you. Which one do you want to order from? Because voice is a powerful way to make specific features of your app quickly accessible, it's important to provide this Add to Siri setup flow in the flows that people already use.

A lot of you developers already do a great job of providing Add to Siri flows in your Settings views, but we all know the best real estate to getting someone's attention is while they're focused on your app to do something. Setting up a shortcut should never interrupt that flow, of course. We're not asking you to sign up for an email newsletter in a pop-up, but providing it somewhere like an order confirmation view can help people get the most out of your app. There is also, of course, the Shortcuts app, which is built right into iOS, iPadOS, and now watchOS.

This is another popular place for people to set up shortcuts for your intents, as well as stringing them together with other intents from other apps to do some pretty powerful things, even automatically.

In the Shortcuts app, intents can actually take in information from other apps, and likewise provide information back to other apps. And the people that use your app can set these kinds of multistep shortcuts by just dragging and dropping in the Shortcuts app.

And with that, you can do multiple things across multiple apps with a single tap.

This may be a somewhat ridiculous example, but it's going to illustrate my point well, so let's go for it.

Here, I have my "order history" intent from Soup Chef in a shortcut.

This intent returns a list of the orders I've placed per week in the past seven weeks.

Below it, I have a great example of a full-featured intent from the App Store, the "new chart series" intent from the graphing app Charty.

This action consumes the output of the Soup Chef action, the weekly number of soups I ordered, and passes them onto Charty to make a graph. Now, when I tap "Run," I'll get a chart of my soup-drinking habits.

Look at that beautiful graph. Wow. That's a lot of soups. Again, this example may not apply to you exactly, but intents and the shortcuts they power are incredibly flexible. Instead of creating a data visualization of your soup drinking, and, yes, you drink soup, you can ask Ari about that, he manages the Shortcuts team, you could just as easily hook up an app that logs your water intake and visualize it using Charty, or almost anything else you can imagine. Also, as Sofiane discussed, we, the system, can learn and predict future behavior, too, using donations. Soup Chef donates its intents as you use the app. After a while, if iOS notices that you order soup when leaving work on Thursdays, we can show its prediction directly in Search.

Or even as an automation suggestion in Shortcuts to automatically order soup when going home. That's intents in Siri and Shortcuts. Now we hand it off to Chad to talk about how system intelligence works with the brand-new widgets in iOS 14. Chad? Thanks, Mert. Hi, I'm Chad, and I work on the Proactive Intelligence team. Today, I'll be talking about the new widgets in iOS 14, and how the intelligence of the system enhances the user experience.

This might sound obvious, but part of what makes apps so great is that they provide people with timely information. For instance, when I want to check the weather for my family in New York, it's as simple as tapping on an app icon, although I might need to scroll through a list to find the city.

Or I can open Music to find some tunes. Let's see. I'll... How about some jazz? Yeah, that sounds good.

Or if I wanna look at my list of very important tasks, I can open the Reminders app and drill into the list.

Just as in these examples, there are some pieces of information that are so important, you revisit them time and time again.

And that's the problem we wanted to solve with the new and improved widgets.

Widgets make it super easy to see the information that's deep in an app's UI right on your Home Screen. It's glanceable, and the best widgets are also timely and relevant.

As you would expect, tapping on the widget brings you right to that exact spot in the app.

Widgets are now backed by intents. Intents allow people to configure the exact kind of data they want shown in their widget. It allows for more personalization. The intent is the kind of widget.

In the case of weather, the "show weather" intent allows you to pick a specific location because the location is defined as a parameter in the intent.

Now let's talk about another new feature: Stacks.

As you can see, it's very easy to drag and drop multiple widgets on top of each other to create a stack.

But we wanted to make sure the topmost widget was timely and relevant using a combination of intents and donations.

And what's really cool is that we use donations to actually rotate the widget that's on top of the stack. So, let's say I check the weather in New York the same time every day, the system will learn that and... automagically it flips for me.

As Sofiane mentioned, donations help us learn how people use your apps.

And as Mert mentioned, that can drive things like Shortcut Suggestions on the lock screen or in Search.

Similarly, the way people use your main app is what drives this automagical experience.

So, not only are we making it easier to see relevant information, if it's something you look for every day, we eliminate swiping around your phone to make it even more glanceable.

It's classic "surprise and delight." If we do our job right, the user may not even know that we're doing all this intelligent calculation under the hood. They'll just see that the information they want is right there and move on with their day.

The less effort required, the more successful we've been. That's all for the new widgets and Stacks. Now back to Sofiane to talk about Siri Event Suggestions. Thank you, Chad. There are many other technology areas that build on the same concepts.

With Siri Event Suggestions, our goal is to make integrating your reservations with the system really easy, and enable people to get a ton of great benefits along the way. This feature is also powered by the Intents framework.

When a user views a reservation, your app should donate the details of that reservation to the system.

And we know you interact with customers through many channels, like your website or even emails. So in iOS 14 and macOS Big Sur, we're introducing new ways to integrate with Mail and Safari by using web markup.

These were just a few of the technologies which leverage these three concepts: define, learn and execute. By making friends with the system, you can meet users where they are, and make it easier than ever for people to get things done.

Now I'll hand it back to Mark to talk about next steps. Thanks, Sofiane. We've hammered home a few key ideas. The intelligent experience we strive for should make the everyday easier for people.

It should help you get more value out of your devices and your apps.

It's about spending your time well, not spending your time going through the motions, even if that motion is a swipe up to Home, or a tap on an app icon, or a drill into a detail view. We want you to experience your life as it is, fully in control, not overly tethered to a device.

Our apps and our devices feel magical not when they force you to engage with them, but when they enhance the things you do out there in the real world.

But we can't do it on our own. It's only an intelligent system if each of its constituent parts works in harmony, speaks the same language and helps people solve problems they didn't even know they had.

As JP put it so well, we want people to feel like they have superpowers. And as developers and designers, you have the opportunity to give that to the people using your apps. So why don't you? You'll find links with resources to other videos, including how to implement the technologies you've seen here. No matter which you choose, just remember to define your key features as intents, donate to the system to power suggestions, design a great voice experience for people to use, but above all, focus on how you can add value and make sure that people's time is well spent. Thanks for watching, and I hope you have a great rest of your WWDC.