The process of inclusive design

♪ Bass music playing ♪ ♪ Caroline Cranfill: Hi, I’m Caroline Cranfill, a designer on the Apple Design team.

Later, I’ll be joined by my colleagues, Cynthia and Sabrine.

People are both equally important and very different from one another.

Considering each person can broaden your market, lead to a more popular app, and is so much more than the right thing to do.

Achieving this ambitious goal will require you to think from the very beginning about how to ensure your app will empower and delight everyone, and not just at the beginning.

As with all projects, designing and developing for inclusiveness is an iterative process that requires you to examine your assumptions about how other people think and feel.

Although each person’s perspective is unique, inclusive design and development benefits from focusing hard on certain axes of diversity.

This can help us think more broadly than our own experiences.

Diversity axes can include class; culture; ethnicity; language; education; political, philosophical and religious beliefs; race; gender; sexual orientation; age; abilities; disabilities; whether you are right- or left-handed; body measurements like height; and also environmental conditions like location, internet connectivity, or access to technology devices.

Each of these provide spectrums of perspectives -- and even more unique experiences -- when you consider the intersections of multiple axes at any given time.

For example, how will your app perform if there is no high-speed Internet or data is limited? Or, will your app be welcoming for people across the spectrum of gender identities and LGBTQ+ communities? As you view your app or game through many different perspectives, inclusiveness is about making people feel welcome and considered.

People who use it should feel that you designed with their life experiences in mind.

Because your app can be used by people around the world, it lends to celebrating humanity and difference.

Today, we will discuss how creating apps or games for a diverse set of people requires inclusivity to be infused into how you work just as much as what you choose to make.

Before we dive into specifics, I will talk through some common misconceptions around inclusive design and development and how you can think differently.

At first, it may feel like inclusion is hard.

But I would like to reframe this as, "Inclusion is a journey." Lots of things are hard, but we do them anyway when they’re worthwhile.

Change is an iterative process and takes time.

Meaningful change requires planning, persistence, and patience.

The benefits are high, because you can improve quality of life, enable people to achieve new goals, or try new things.

Another misconception is that inclusive design limits creativity.

In truth, inclusive design and development stimulates new creative solutions.

Wider context and awareness can promote progress in areas that you may have never considered before.

You can make breakthroughs in experiences for everyone that wouldn’t happen if you didn’t consider the diversity of human experience.

Inclusion doesn’t mean turning down innovative ideas or asking all people to accept a lowest common denominator experience.

Constraints help your process by helping you make the first few early decisions.

This allows you to put most of your energy into creating new solutions in a narrow problem space.

Let me share an example with you.

A team at Apple figured out how they can use our Apple products to add a layer of intelligence to our physical world.

They first wanted to focus on recognizing a few objects and text, which in itself is a constraint.

But they didn’t just stop there.

They explored ways to surface buttons to call, message, add to contacts, or copy recognized phone numbers.

It’s helpful to me but essential for people who are blind or low vision to interact with printed materials like signs, business cards, and envelopes.

Not only can people who are blind read this text, they can act on it to make calls and save contacts.

This brings us to a misconception about teams.

You may think that having a diverse team means your app or game will be inclusive.

In truth, having an empowered diverse team means your app or game can be inclusive.

We all share the responsibility to build an inclusive environment that enables inclusive actions.

You’re more engaged when you feel comfortable voicing your ideas and opinions.

And when those thoughts come from a diverse group of people, it fosters more creativity and innovation.

At Apple, the accessibility engineers work hand in hand with a wide array of teams to help solve unique problems that affect only a certain group of people.

For example, a person who is blind or has low vision might have difficulty knowing when to move forward in a line independently.

A solution started with the video engineering team using the technology called "LiDAR." It uses a pulsed laser to ping nearby objects to generate a 3D model of the surrounding environment.

The teams were empowered to work together to create and release the People Detection feature in the Magnifier app.

It allows you to know the distance between you and the people around you.

What started as a prototype to make life more convenient became a necessity once COVID-19 hit.

Our team's ongoing conversations with the blind community led to a more pressing issue: How does a blind or low-vision person practice social distancing? One more misconception is that inclusion can be done at the very end.

Considering inclusion and the experience that each individual person -- or groups of people -- have with your app or game must be considered upfront and carried throughout the design and development.

Waiting until the end is reactionary, and there often isn’t enough time to correct.

More than small changes become risky.

This doesn’t mean you have to get it right from the start, but that it's being considered from the start.

Now we can build on these reframings to empower us to create more inclusive apps or games.

To start, you can take a deeper look into how you work.

You can be in a better position to achieve your inclusion goals if you first look at the diversity of your teams in creating inclusive working environments.

At Apple, we believe our teams should reflect the diverse, ever-evolving world we serve.

People are drawn to like-mindedness, consensus, and agreement.

It feels comfortable and takes less mental energy, but it doesn’t always get you the best results.

When you see difference as an asset, rather than a challenge, you open yourself to new possibilities, creative thinking, and productive new relationships that help you make better decisions.

In some ways, small companies are better positioned to build more inclusive teams if you commit to diversity early.

People tend to hire those they know, which poses the risk of them having very similar perspectives.

If you instead bring on someone different from you, you have an easier path to making diversity a core value from the get-go rather than rushing to fix it later on.

And as you grow your teams, think “culture add,” rather than “culture fit.” What perspectives are you missing and how could finding that perspective change your app for the better? Your team can be diverse, but it’s often that diverse perspectives are not incorporated in meaningful ways that impact decision making.

You want to make sure you are actively creating space and asking for perspectives in a judgment-free way.

To create safe spaces, teams need time and space to think inclusively.

Protect this time.

The design and development process can be rigid and unwavering, so much so that you don’t feel like you can take time to reflect and think about what you are making.

Fast development also doesn’t allow you to incorporate the diverse perspectives from your teams.

Now we will look at the design and development process more closely, and I will share tips and pointers to consider during each phase.

If you were to illustrate the process, you may first think of it as a line with point A of formulating the idea to point B of releasing.

In the middle are lengths for phases like design, development, and testing.

Much like security or privacy, inclusion cannot be treated as an afterthought.

Since you don’t want to release your app or game and discover that it doesn’t work well for everyone, you can dedicate and reserve time to pause, think, and pivot throughout the whole process, not just at the end.

It really becomes a rejuvenating, iterative process.

Taking time out of each phase to reflect, ask some key questions, can be helpful to evaluate what you are making.

It may feel like you are looping back momentarily, but the new, more inclusive path will ultimately make the app or game relate to more people.

It will also give you more time to tackle things that may be complex and require longer time to solve.

Now we will discuss what you can ask yourself or do during each phase.

First, we’ll start when you ideate on what you are going to make.

Everyone working on the app or game should be able to answer the question: Why are you making this? Will everyone relate to it? Clarify the purpose of the app or game you are creating.

What human good are you trying to serve? How will people be better off? What does it enable people to do, be, or feel? Next, also ask, who are you making it for? Don’t just think of people who use the app or game currently.

Think through the lens of the different diversity axes.

Which specific dimensions or intersections have you considered, and what will their experience be? Next, challenge your assumptions and listen.

Actively listen and learn about the experiences and perspectives very different from your own and those commonly valued by your design and development team.

Your educational journey will uncover new areas to concentrate on.

The next phase of the design and development process is design.

When ideas are coming to focus, it’s time to think concretely about the extremes of how people will use the app or game, not just the average person.

It may be that an unconsidered intersection is preventing people from wanting to use your app or game.

Next, consider the emotional, social, and physical impact on the people who use your app or game.

Are some populations or groups of people more likely to have negative experiences? Are these populations also vulnerable or disadvantaged? If so, do you understand why, and what is being done about this? Next, consider the impact of location and context on the experience.

Context is about the circumstances of the people using your app.

It can span across several diversity axes like culture, environment, location, and connectivity.

You can consider how in some cultures, multiple generations of families may live together in one household.

People may live in cities or rural areas, and both have different needs.

You can consider environments may be noisy or quiet, bright or dark, or small or large.

For example, roommates or friends using an AppleTV may have a different experience and needs from families with children who use an AppleTV.

The next phase of the design and development process is when you start to develop your designs.

It’s important to have inclusive representation or relevant expertise throughout development.

As you evaluate prototypes or designs, be sure to explicitly include a diverse group of people when solutions are being discussed and decisions are being made.

Diverse voices can provide context and strengthen reasons on why the app or game is going down a certain path.

Next, start planning for accessibility and internationalization at the beginning of development.

Apple ships products with lots of built-in accessibility features for people with disabilities, who are 15 percent of the world’s population.

And the number of people living with disabilities is increasing due to aging populations and chronic health conditions.

The people who use your apps or games need you to make them accessible.

Take time now to understand different accessibility and localization APIs, so everything is in place as features are developed.

Standard UIKit, AppKit, or SwiftUI controls and views implement the UIAccessibility methods and are accessible by default, but you will need to plan for more time to make your apps accessible if you are using custom controls or views.

Before wrapping up the design and development phases, I’m going to hand it over to Cynthia to share an example of considering intersectionality in innovation.

Cynthia Bennett: I’m Cynthia Bennett, and I’m a researcher on the AI and Machine Learning team.

Our team designed and developed one inclusive experience -- powered by machine learning -- which centered a perspective at the intersection of two of the diversity axes.

And I’ll explain how learning from organizations outside of Apple helped us to develop a better feature.

To provide some background, people who are blind or low vision access Apple products with our advanced screen reader, VoiceOver.

With VoiceOver, people who use our devices can navigate apps, chat with friends, and do much, much more.

Let me show you how it sounds.

While using VoiceOver, you can drag your finger across the screen and hear everything you touch.

VoiceOver: VoiceOver On.

Calendar, Monday June 7th.

News. Home. Camera. Podcasts.

You can also flick left or right to move from one element to the next.

VoiceOver: News. Weather. Photos.

And you can double-tap to activate items.

VoiceOver: Double-tap to open. People who are blind or low vision may not be able to see what is in Photos, but they want to enjoy them just like everyone else.

Image descriptions -- or textual information about the most important things shown in photos -- may accompany photos.

So to create great and inclusive photos experiences, Apple has worked on supporting image descriptions.

In iOS 15, you can use the Description feature of Markup to write your own descriptions of photos.

These image descriptions are then sent whenever you share photos across Apple products.

The human-written description for this photo is, “Side-profile of a giraffe walking in the savanna with trees in the distance.” But there are a lot of photos out there and many of them do not have descriptions.

Machine learning is a tool that can help.

A machine-learning-trained model can generate image descriptions automatically, quickly, and at scale.

So our team trained a model to generate detailed natural language descriptions of images across iOS, which we call VoiceOver Recognition image descriptions.

Let’s listen to the VoiceOver Recognition image description of this photo.

VoiceOver: A giraffe standing on a dry, grassy field.

But even state-of-the-art machine learning is not enough.

Inclusion must be intentional.

So our team needed to understand how the image descriptions model described all people, not just average people.

If you’re working on machine-learning-powered features, one place to start is to learn how different groups of people are represented by machine learning.

And this is a step everyone can take, whether you’re working on your own or at a large company.

Keep those diversity axes in mind when thinking about what groups you should prioritize.

One aspect of machine learning representation that our team learned more about was how people’s gender would be described.

We learned that so far, models tend to represent two genders: female and male.

But we know that people may be a variety of genders, including being non-binary or transgender.

In this case, representation did not align with everyone’s gender.

You might ask, why does this matter? What does gender have to do with making photos inclusive of one group -- people who are blind or low vision? Well, people who are blind or low vision have all sorts of gender identities.

In other words, these characteristics will work together to impact the experiences people have while reading image descriptions.

Intersectionality was developed by legal and critical race scholar Kimberlé Crenshaw.

It recognizes the unique experiences and needs of people who have multiple identities which are systematically disadvantaged.

In this example, listening to intersectional perspectives helped us to develop great experiences for people with identities across groups in the diversity axes.

Specifically, intersectionality helped us listen to people tell us how being both blind or low vision and being a minority gender, like non-binary and transgender, together impacted their experiences reading image descriptions.

To ensure that we learned perspectives from people who live at this very specific intersection, our team partnered with outside organizations with members who are both blind or low vision and who are part of the LGBTQ+ community, which is inclusive of non-binary and transgender people.

Our team learned that you can’t know someone’s gender by looking at a photo.

We also learned that gender-neutral image descriptions could still provide helpful details by focusing on other things, such as people’s clothes and hairstyles.

For now, we developed VoiceOver Recognition image descriptions to be gender neutral.

As an example, let’s hear the VoiceOver Recognition image description of this photo.

VoiceOver: A person with curly black hair wearing a red and white striped shirt and standing in front of a building.

However, anyone can use the description feature to write their own image descriptions, which can include people’s gender when the person writing the image description knows this information.

A human-written description for the same photo could be, “Sam stands outside of a restaurant.

She has curly black hair and wears a red and white striped blazer.” So we just looked at the process of inclusive development as it applied to using machine learning.

But the things I talked about apply to any app or game.

First, inclusion is innovation for wider audiences.

For example, image descriptions enhance engagement for everyone by allowing people who are blind or low vision to take part in all of the fun we have by sharing photos with one another.

Even if you don’t have the resources to train machine learning models to increase accessibility, you can still create great experiences with features that enable people to be more inclusive, like our description feature.

Second, including people who live at the intersection of various identities and experiences helps you to not just develop for one group at a time, but to make sure your products intended to meet unique needs, like those of people who are blind or low vision, are also respectful across diversity axes, which we did by developing image descriptions that are inclusive of all genders.

Our team learned about this important intersection by researching how machine learning represented people.

And this is something anyone can do, whether you're working on your own or at a large company.

Finally, reaching out to experts when we did not know something, was a key component of our inclusive process.

Caroline: As Cynthia discussed, focusing in on intersections of diversity axes is one way to not only be innovative, but respectful.

To continue discussing tips for inclusivity in the design and development process, the next phase is when you test your app or game.

We need to seek out and engage with a diverse group of people to use your app or game to give real-life feedback.

Most of all, you’ll discover whether the ideas in your mind hold up in practice and over time, as well as gain extra insights.

Having testing teams with diverse backgrounds can help find those edge cases.

Diversity will bring a wider range of experiences and usability patterns.

Next, testing with assistive technologies and various accommodations will help you uncover issues before they affect customers.

Learning the basics of VoiceOver, increasing your text size to the maximum level, or navigating your app or game with the keyboard can help you quickly identify potential problems and should be incorporated into test plans.

Next, listen, review, and prioritize feedback.

Feedback is important to the final product.

With feedback, you may need to repeat the design and development process with new lenses, remove features, or add features.

Advocate in your company to put inclusive testing goals as Key Performance Indicators, or KPIs, just like any other metric, to measure inclusion as a key business objective.

The last phase of the design and development process is deciding if you are ready to release.

At some point, given time and resources, you will need to make tradeoffs to release if you have a hard date.

It may mean you need to make hard calls by cutting back the release of one feature so you can make fewer features great for everyone.

Poor experiences for a particular group of people can have severe impact if it continually happens over and over.

It can lead them away from your app or game.

Next, if you can’t have the best experience for one of your diversity axes or intersections, you need to take time to know why, how, and when you can plan to make the best experience.

Sometimes not enough time was allotted, an issue was identified or escalated too late, didn’t plan for additional complexity, or perhaps the API was novel and you ran out of time.

But having an informed and properly scoped plan and target date to address improvements is important to stay on track and to be able to plan better next time.

Learning how to do better and how we can do more is part of the inclusive journey.

You can also remember to continue to reevaluate, strengthen, and grow with real-life developments, or as you learn more about the ways you can make the experience better for people.

It also means not just moving on to the next thing entirely.

Inclusivity allows you to act towards massive change.

To walk us through an example of continuing to iterate designs to be more inclusive, I’ll hand it over to Sabrine.

Sabrine Rekik: Hello, I'm Sabrine Rekik, an inclusion engineer in the Camera & Photos team.

My multicultural upbringing taught me that empathy and curiosity make the world a better place.

I will share insights and techniques on how to continue your inclusivity journey after you launch.

To do so, I will use the example of a feature I worked on, called photo Memories.

In case you are not familiar with it, photo Memories smartly stitches people's photos and videos into heartfelt thematic movies and help them revisit forgotten moments.

In iOS 15, we wanted to expand the photo Memories experience to be even more inclusive to our global audience.

To guide us, we paid close attention to the feedback shared by people using the Photos app.

Many people said they enjoy photo Memories and being reminded of good times with family and friends, but others hinted at ways Memories could be improved.

I will show you two pieces of feedback we received, and how we turned them into inclusion goals.

The first comment is, "I take a lot of photos at my daughter’s karate practice.

It would be great to have them turned into a Memory.” We turned this feedback into the following goal: celebrate diversity.

How? By surfacing memories people care about, with themes that match who they are and what they are into.

To come up with those themes, we organized brainstorming sessions with our teammates throughout the company, especially those living outside of the US.

Brainstorming is my favorite group technique to quickly and spontaneously generate ideas and boost creativity.

We gathered unique ideas to make Memories more personal; many of them we would not have thought of by ourselves.

We were reminded that even though many of us love taking photographs, the content of our photo libraries is unique to each of us.

Our photos and videos reflect our own identity and life story, including our hobbies and cultural heritage.

To help people relive moments they care about, we had to embrace the breadth of life experiences and interests.

Little tip for your brainstorming session: identify voices that are very different from yours.

And if possible, run multiple sessions with smaller groups in different time zones to allow more people to participate.

Large groups tend to be intimidating, and you can miss out on great suggestions.

Once we gathered new memories ideas, we analyzed each one through the lenses of the diversity axes.

In our case, we looked at new Memory themes through the lenses of: rural versus urban living, teenagers versus grandparents, regional versus global.

And for each of them, we answered the questions: Who are we building this for? Who are excluding? And how will people feel? This exercise helped us select and prioritize new Memory themes.

People wanted more of their hobbies to be included, so we started representing more of them such as martial arts, skateboarding, and soccer.

We chose to prioritize hobbies around the world, rather than hobbies in our areas.

But we didn't stop there, we also expanded our international holiday catalogue so every person can relive festivities they cherish, whether it be Christmas, Diwali, Lunar New Year, Eïd al-Fitr, Hanukkah, or all of them.

Since my team didn't have the expertise required, we collaborated with cultural experts to identify the set of holidays people living in their regions love celebrating.

I'm thrilled that the Photos app can help me revisit even more aspects of my life, including my favorite Tunisian, French, and American holidays.

Now let’s look at the second feedback we heard.

Someone commented, “I don’t like when the Photos app reminds me of my ex-partner.” We turned in into the following goal: Being mindful of difficult life experiences, and design with sensitivity.

To achieve this objective, we used another technique.

While brainstorming meetings are great at producing ideas and increasing creativity, specific topics require extra sensitivity and your undivided attention.

So we leaned towards individual interviews to dive deeper and provide a safe space for people to share their stories.

I encourage you to give it a try and schedule conversations with people who had concerns with your app and didn't feel included.

For Memories, we hosted several conversations to unpack the feedback gathered during our preliminary research.

Several people shared that they enjoyed revisiting their photos, except when they featured a person they'd rather not be reminded of.

Before designing solutions, we carved space to empathize, listen, and understand their perspectives.

We gently unpacked the what, how, when, and especially the why.

We were interested in what made a memory enjoyable or not, but this question is difficult to answer.

To avoid vague and generic replies, we asked instead: What are some of your favorite Memories? What Memories did you dislike? What were you feeling after seeing it? How can we make the experience better for you? And followed up with things like, “Why?" or "Tell me more about that..." to unpack the answers even further.

Formulating the right questions can be challenging, but it’s a skill you get better at with practice.

If you can, favor open-ended questions, refer to specific moments in the past and avoid using language that can influence the response.

Also restrain from bringing solutions to their problems just yet, and take things in.

By having one-on-one conversations, we were reminded that photos and videos have the power to affect someone's emotions, both positively and negatively.

It can be difficult to unexpectedly see photos of a loved one who passed away and be reminded of their absence.

For others, pictures of their ex-partner can elicit a myriad of emotions, from sadness to anger.

And we also learned that sometimes, it's not what is in the frame that hurts, but the invisible context that only the owner of the photo library knows about.

The process of listening and absorbing other people's experiences can be humbling.

It was for me and my team.

After being immersed in real stories, it is now time to address the feedback with a new design.

In our case, to help people revisit photos and videos that bring joy, we thought we'd give them control over who they see, or not, in Memories.

In the first iteration, we added multiple buttons to feature less of a person while browsing a Memory, and we reengineered our app, so they’ll immediately see less of these people across the Photos app.

For consistency, we looked at other places in our app where people would want to affect the experience.

We added a Feature Less action from a Featured photo, from the People albums, and even from a single asset while browsing.

The next step is to validate that a solution to a problem works.

As designers and developers, the first design we come up with is our first response to the stories we heard and absorbed.

However, listening to someone doesn't always mean we understood them.

To make sure you are on the right track and solve real problems, it is important to circle back to the people you interviewed, show them your sketches, and seek their feedback.

Is your design addressing their need? And if not, try to understand the nuances you missed the first time.

When we showed our first solution to people, we got mixed reactions.

We learned that we all have different tolerance thresholds and that the action of featuring less of a person wasn't radical enough for some.

We took this insight into account, and in the following design iteration, left it up to each person to decide what they would like: feature a person less, or never feature it again.

It might feel evident in hindsight.

However, it happens to all of us.

When we get too close to our products, we develop oversights and fail at meeting people's needs.

But empathizing with people can help us course correct when we didn't get things right from the first try.

Lastly, even details like glyphs and icons can have an impact on people.

During the final phases of development, we observed that the thumbs down icon could project a feeling of disliking the person.

This might be the case in some situations but not for others, like when a loved one has passed away.

So we decided to use a more neutral glyph instead.

So, we just saw through the example of photo Memories that launching a product is not an end, it is a new beginning in your design and development journey.

By having inclusion in mind, you will continue to make your experience better for everyone.

Remember to first, look at reviews and feedback people using your app or game left on the Internet, and go the extra mile to seek feedback from people with experiences different from yours and your friends.

Second, empathize and put yourself in other people's shoes by using the many techniques at your disposal like brainstorming, working with experts, and individual interviews.

And finally, address the feedback even when it’s not your own experience with the product.

And remember that sharing back solutions for validation can help ensure the experience has actually made a more positive impact.

Now back to Caroline.

Caroline: We hope the diversity axes and the examples we have shown you today are useful as you tailor an inclusive design and development process that fits your company and context.

To conclude, inclusion is a journey.

Yes, the journey is sometimes hard, but we love solving hard problems, right? Difficulties should only drive us to try harder to make our apps and games more inclusive.

Inclusive design and development stimulates new, creative solutions.

Inclusion does not deliver subpar experiences, but instead opens up opportunities in areas we may have never considered, and offers productive constraints to narrow our scope and design with more intention.

Empowered diverse teams can deliver more inclusive products.

Diverse representation does not guarantee inclusivity will come naturally.

Instead, when people are invited to share their perspectives, and when perspectives together impact decisions, products can be more inclusive.

Finally, inclusion must be infused into the entire design and development process -- into empathizing, ideation, designing, development, testing, release, and beyond.

The examples we shared demonstrate that inclusion is as much of a process as design and development are, and the resulting features are better because of the attention given to inclusion.

Listening to the lived experiences of people will help you to understand the impact of your app or game, and to release both innovative and respectful products in response.

Keep the diversity axes and their intersections in mind as you choose who to focus on.

We can practice empathy and know that inclusive design is the right thing to do, but it’s time to just call this "good design." We can authentically connect with our audiences and learn new perspectives.

We can handle feedback with grace and design with them, rather than for them.

Let this be day one if you haven’t started this journey yet.

You just have to start somewhere and try to do a little more everyday, which helps it become habit and then the new normal.

Be sure to check out “The practice of inclusive design” to learn methods for designing content so that your user interface, language, imagery, and marketing materials support the broadest possible range of people.

♪