Optimize CarPlay for vehicle systems

Hi, I'm Kartik, from Car Experience Engineering, and welcome to "Optimize CarPlay for vehicle systems." Today, we're thrilled to share how your vehicle can deliver the best CarPlay experience to drivers.

If you're an automaker or developer of vehicle systems, this is the session for you.

CarPlay is the smarter, safer way to use your iPhone in the car. A large part of what makes a great CarPlay experience is how effectively you integrate CarPlay in your vehicle. Today's session consists of 5 topics to help you achieve this.

Visual integration is all about making CarPlay look and feel natural in your system. Then we'll cover CarPlay infrastructure advancements in connectivity, audio, and video encoding, which all serve as the backbone of a great CarPlay experience.

And in our final topic, we'll go over EV routing in CarPlay.

Last year, we gave a sneak peek of the next generation of CarPlay. The infrastructure advancements we're introducing today will help prepare your system for the next generation of CarPlay.

Let's get started with visual integration features.

Modern vehicles feature large main displays with an increasing variety of shapes and layouts. Some displays are wide, while others are tall. Some displays are non-rectangular, and some show it all, by featuring a mix of CarPlay UI and built-in vehicle UI. CarPlay is designed to integrate well into all these variations of main displays. Let's see how you can tailor CarPlay to make it feel at home in your vehicle.

In a previous WWDC video, we introduced the CarPlay view area. Today, we're building upon this concept. The view area defines the boundaries where CarPlay draws its UI.

CarPlay looks great in full screen. However, your built-in vehicle UI may have additional layouts where CarPlay appears alongside widgets or other elements.

In this case, support dynamic screen resizing so people can seamlessly switch between layouts.

If your system displays a widget next to CarPlay, CarPlay automatically insets itself to create more visual spacing with your built-in UI. You don't need to inset the view area; simply give it as much space as possible, and CarPlay will create the right amount of padding. What if your display is non-rectangular? In this case, define the view area as the smallest rectangle enclosing your display. Also, define a safe area as the largest rectangle within your display.

CarPlay draws interactive content inside the safe area to ensure all important UI is visible. Outside the safe area, the pixels are black. Now you have the option to enable CarPlay to draw UI outside the safe area. CarPlay will draw its background to the edge of your display, giving the driver a visually appealing and immersive experience. Keep in mind that this safe area flag is only available for the main display.

Let's turn our attention to the CarPlay status bar. The CarPlay status bar is permanently displayed in CarPlay's UI, providing the driver with convenience at all times. The status bar is intelligently positioned based on your display's resolution and aspect ratio. It can appear vertically, on the driver side, or horizontally, along the bottom edge of CarPlay's UI.

In some cases, it may make sense to override the status bar's position for a better fit with your system's built-in UI.

This can be done with the view area flag. For example, you may want to align the status bar with other permanent controls in your system's UI. The status bar also features some slick rounded corners.

We love how rounded corners look on CarPlay, and corner clipping masks help with their visual appeal.

Let's start with some context. By default, the background of CarPlay's rounded corners appear in black. This looks great when CarPlay is full screen.

Here's a system where CarPlay is drawn in a window. If you look at the corners of CarPlay's UI, you can see the black background showing behind CarPlay's rounded corners. These small black triangles now appear which may not blend well with your system's wallpaper. By supporting corner clipping masks, your system can magically make these black triangles disappear.

The way this works is, iPhone provides a corner blending mask containing transparency information.

Your vehicle system applies this mask to each corner of CarPlay's UI to remove the black triangles. Then you can draw your background all the way up to CarPlay's rounded corners. This makes CarPlay truly blend into the built-in system and create great visual appeal.

Take note that corner clipping masks cannot be used when CarPlay draws outside the safe area. It's time to shift our focus onto the next feature, UI focus transfer.

Some systems support a rotary knob or touchpad as an input device.

As the driver interacts with CarPlay using this input device, CarPlay UI shows a focus highlight on the selected element.

Your system may show its own focus highlight as the driver scrolls through the built-in UI.

CarPlay supports focus transfer, allowing the driver to seamlessly move between UI elements in CarPlay and your system.

The driver can simply nudge the knob to transfer focus.

Our APIs ensure that only one focus highlight is shown at a time by arbitrating focus between CarPlay and your system.

Support focus transfer if your system presents CarPlay in a windowed configuration and supports a knob or touchpad.

Let's walk through how focus is coordinated using the CarPlay protocol.

When CarPlay starts, the built-in system determines if focus should be given to CarPlay.

Here, you can see focus belongs to the system.

When the driver directs focus towards CarPlay, the system dismisses focus and provides heading and position information to CarPlay.

CarPlay uses this information to present its focus highlight on the most intuitive UI element.

When the driver directs focus back to the built-in UI, CarPlay offers focus to the system. If the system acquires focus, CarPlay dismisses its focus and the system presents its own focus highlight.

Adopt UI focus transfer to create continuity between CarPlay and your built-in UI.

Let's talk about CarPlay appearance mode.

CarPlay UI can appear in either light or dark themes. This brings the familiar light and dark UI experience which people love on iPhone.

Synchronize CarPlay's appearance with your built-in UI appearance. People will appreciate seeing CarPlay and your system in harmony.

CarPlay's appearance can change based on vehicle state, user settings, and time of day. For example, if your system shifts to a dark theme at night, your system simply informs CarPlay to change appearance based on time of day.

And if your system activates night mode, CarPlay will automatically shift into a dark appearance.

There is also a separate appearance mode control for Map UI. Let's walk through a few examples to illustrate how all this works.

In this example, the built-in UI is using a dark theme on the main display.

The system informs CarPlay to use a dark appearance mode to create a consistent look.

If your system shows CarPlay navigation UI on the instrument cluster, specify the appearance for each display.

In this case, both the main display and instrument cluster are in a dark UI theme.

If your system shows different appearance modes per display, inform CarPlay to match. And lastly, CarPlay's map UI has a separate appearance setting. Keep your eyes on the instrument cluster as we shift into a light map appearance.

Be sure to synchronize CarPlay's map UI appearance with your built-in system's map appearance.

Now that we've covered synchronizing UI appearance with your system, let's look into what kinds of content iPhone can power across your system's displays.

We'll be covering three common kinds of displays in modern vehicle systems: the main display, the instrument cluster display, and a head up display. Let's review the iPhone content available today for each of these displays.

On the main display, show the CarPlay video stream. In addition, your system’s built-in UI can be powered with iAP2 metadata like route guidance, phone call, and now-playing information. If your system presents a map widget, populate the widget with a CarPlay navigation UI stream while CarPlay navigation is active.

This UI stream can be configured to present ETA information, speed limit signs, and a compass.

On the instrument cluster, you can show CarPlay navigation UI streams.

If your system has controls to change the map zoom level on the instrument cluster, add CarPlay map zoom support. And implement your built-in UI with iAP2 metadata as you would on the main display.

On the head up display, use iAP2 metadata so people can view information like turn-by-turn directions in their line of sight as they're driving.

Let's look under the hood at infrastructure features to create a reliable and performant CarPlay system.

Ultimately, these changes will also set your system up for the next generation of CarPlay. Our first topic is all about connectivity between iPhone and your vehicle system. The wireless connection experience begins when a driver pairs for the first time.

Make the first-time-pairing experience easy for drivers by supporting out-of-band pairing.

Out-of-band pairing allows people to pair their device for wireless CarPlay by simply plugging their iPhone into the car. It's convenient and requires less interaction. People expect all wireless CarPlay systems to support out-of-band pairing over USB.

If your system supports digital car keys, we've got something new for you.

Starting with iOS 17, iPhone supports pairing for CarPlay over a car key connection.

This makes pairing even easier for people, as car key and CarPlay pairing experiences combine into a unified flow.

After adding a car key on iPhone, people are prompted to pair for wireless CarPlay. If accepted, pairing occurs over the car key connection, and CarPlay connects wirelessly.

After performing a one-time pairing, your vehicle automatically connects with iPhone every time the driver enters the car. Bonjour is used for your vehicle and iPhone to establish an IP connection and start the CarPlay session.

We've introduced a simplified connection flow which is the preferred way to connect CarPlay.

By using the existing iAP2 connection to perform IP address and port information exchange, Bonjour is no longer needed, and the implementation is simpler and faster.

This also adds support for WPA3-only networks. The simplified connection flow can connect with devices running iOS 14 or newer. If your system is designed to connect with earlier iOS versions, continue to support the Bonjour connection flow. For those of you who support car keys, we're back with another feature.

Your system can instantly connect to wireless CarPlay as soon as the driver enters the vehicle with a digital car key.

Using iAP2 messages, your system can map a digital car key pairing to a CarPlay device. The next time a driver approaches the vehicle and a car key connection is created, your system prepares its CarPlay stack.

By checking for a corresponding CarPlay pairing with that car key, you can start CarPlay even earlier.

To the driver, CarPlay appears instantly. We are so stoked for you to make use of this in your next system. And let's not forget, we all agree on how important a robust connection is to create a reliable CarPlay experience. To maintain a robust CarPlay connection in your system, design for handling wireless interference, as interference may impact Wi-Fi performance. The first step is to detect interferers on the wireless channel. Once interferers have been detected, steer clear of the interference by using channel switch announcements.

If your system supports multiple wireless radios, you can use BSS transition management to direct iPhone onto another access point. Keep in mind that people best experience wireless CarPlay when operating on the 5 GHz band.

And finally, to create a smoother experience during intermittent disconnects, your system should suppress short link disconnects on CarPlay TCP sockets. Do not close these sockets due solely to a data link layer disconnection.

After connecting to your system, CarPlay can start playing audio through the vehicle's speakers. Let's go over some audio features that are available to your system. Audio apps are now adopting AirPlay enhanced audio buffering to provide a greater playback experience on iPhone. People would love to experience the same audio improvements in your vehicle as well.

Now, you can add support for enhanced buffering to your system, and your vehicles will automatically take advantage of these benefits.

Enhanced buffering is the preferred platform for streaming audio like music to your car's speakers. The audio is provided as an additional stream to the vehicle system, called main buffered audio.

The CarPlay communication plugin contains an up to 2 minute audio buffer, where audio from iPhone is streamed in faster than real-time speeds.

This makes for improved responsiveness, and audio content can continue playback through an intermittent disconnection.

You may want check out "Tune Up Your AirPlay Audio Experience" for more details on what's available to audio app developers. People expect audio mixing in CarPlay to work as it does on iPhone.

Ensure your system can mix various audio output streams to create a consistent audio experience.

Main audio, main buffered audio, alternate audio, and auxiliary audio can be active and mixed simultaneously.

Enhanced Siri brings the familiar Siri experience on iPhone to CarPlay. People are accustomed to saying "Siri" in order to activate Siri, and they expect it to work in CarPlay as well.

Ensure your system supports Enhanced Siri.

Enhanced Siri leverages the vehicle's microphone to reliably detect when the driver is activating Siri by voice.

Also, audio apps and radio are mixed into the background of Siri audio. People love the resulting uninterrupted audio experience.

And lastly, it enables the driver to instantly activate Siri upon pressing the push to talk button.

See "Advances in CarPlay Systems" for a detailed look at how Enhanced Siri can work in your system. Let's move on to video encoding. Historically, CarPlay UI streams have used H.264 encoding. CarPlay now supports HEVC encoding for all UI streams. HEVC is more efficient and enables CarPlay to support higher-resolution displays.

Your system should continue to support H.264 for backwards compatibility. All of these connectivity, audio, and video encoding features will make the experience so much better for drivers.

For that reason, they are required for systems supporting the next generation of CarPlay. We have just enough range to cover our final topic: EV routing. Apple Maps can help you plan trips that include stops for charging your electric vehicle.

When you get driving directions, Maps analyzes the route and other factors, and identifies charging stations along the way. This year, in addition to support for offline maps, EV routing has some great updates.

Users can now tell Maps which charging network they prefer and see real-time charging station availability on the map. To support EV routing, Apple Maps needs to know about some characteristics of your electric vehicle to tailor the EV routing experience.

In addition, you will need to support interfaces to communicate your vehicle's state of charge. If you have an automaker app, support SiriKit intents so people can use EV routing when inside or outside the vehicle. In addition, you should support iAP2 messages for real-time state of charge when driving. We've made it through all the ways you can enrich the CarPlay experience in your vehicle system.

Soon, you may want your next vehicle to support the next generation of CarPlay: the ultimate iPhone experience for the car.

It brings a fully integrated, immersive iPhone experience to your vehicle.

The next generation of CarPlay builds on CarPlay and fully utilizes the infrastructure changes we covered in this video. Prepare your vehicle for the next generation of CarPlay by supporting these features in CarPlay today.

The features we discussed today are available in the latest CarPlay specification and communication plugin. For more information on developing CarPlay systems, visit developer.apple.com/carplay.

We're excited for you to make the most of what CarPlay offers in your next vehicle system.

Thank you so much for joining us today, and keep riding on with CarPlay.