Supporting New Game Controllers

Hello, my name is James Kelly, and I'm a Software Engineer on the Game Technologies Team at Apple. In this video, we'll be talking about the new features and changes coming to the Game Controller framework this year to help you adopt newly supported game controllers.

The Game Controller framework has been making it easy to add support for MFi Game Controllers to your games on iOS, tvOS, and macOS. By abstracting controller hardware through a common API, the Game Controller framework lets you write your code once without you having to worry about how controller data is mapped.

And this year, we're excited to announce new support for some fantastic controllers. First is the Bluetooth-enabled Xbox Wireless controller from Microsoft. This will now seamlessly integrate into any Game Controller framework-enabled game. And that's not all. The DualShock 4 Controller from Sony is now supported by the Game Controller framework. It will just work with any games that already have controller support through the Game Controller framework.

Many developers have already been creating some great games with controller support. Rayman Adventures, Transistor, Sky Force Reloaded, and Alto's Adventure are some great examples, and just a few of the games on the App Store that support controllers. By using the Game Controller framework, games will automatically support the Xbox Wireless and DualShock 4 Controllers in iOS 13, tvOS 13, and macOS Catalina.

In this video, we'll talk about how to grab a reference to these new controllers through the Game Controller framework, how to access their inputs, some UI best practices for probably supporting MFi, Xbox Wireless, and DualShock 4 Controllers, and our recommendations for how to update your games on macOS, if they previously supported Game Controllers through lower level frameworks such as IOKit. First, let's talk about changes coming to the Game Controller framework in support of these new controllers. As a quick reminder, games that already use the Game Controller framework will gain support for free, meaning if your game supports MFi controllers, it will also automatically support the Xbox Wireless controller and the DualShock 4. The controller's inputs can be accessed via a GCController's GCExtendedGamepad profile.

For those of you who are new to the Game Controller framework, let's talk briefly about how to detect a controller. Each controller is represented by a class named GCController. It's the same class for all controllers, MFi, Xbox Wireless, and DualShock 4 controllers, as well as the serial remote, are all instances of GCController.

The first thing you want to do is get a list of currently connected controllers. To do this, use GCController's controllers class method. It returns an array of GCController instances representing all connected controllers. The array will be empty if no controllers are connected. You should check this array when your app launches and set up the controllers as appropriate.

Now it's common for controllers to connect and disconnect while your app is running. For example, the player may turn on their controller after launching your game, their controller may run out of batteries, or they may just take the controller out of range.

To be notified of these events, add observers for GCController did connect notification and GCController did disconnect notification. Your application did finish launching with options method is a great place to do this.

So, now that you've grabbed onto an instance of a connected controller, let's talk about the buttons and other inputs it may have and how to access them.

First, let's recap what inputs are available on a controller. There are two triggers, a directional pad, two thumbsticks, a menu button, four face buttons, a programmable 4-LED strip, and two shoulder buttons.

Let's take a closer look at how physical buttons map to the API using the four face buttons as an example. Here we can see a table listing the four face button properties found on a controller's extended gamepad profile. The face buttons of MFi controllers map directly to this API as do the face buttons of an Xbox Wireless controller.

Note that there is some subtlety to consider with DualShock 4 controllers as they use symbols instead of letters to represent their face buttons. In cases where there may be ambiguity when mapping an input to API, we use the notion of positional equivalents to resolve these mappings.

For example, if we consider the face buttons to be four buttons arranged in a diamond on the right side of a controller, then the bottom cross button is positionally equivalent to the bottom A button of an MFi controller. So, it maps to the button A property. The right face button with a circle button maps to the button B property and so on.

Next, let's talk about clickable thumbsticks or L3 and R3.

We added support for these inputs to the Game Controller framework in iOS 12.1, tvOS 12.1, and macOS Mojave 10.14.1. While the Xbox Wireless and DualShock 4 controllers all have clickable thumbsticks, thumbsticks on some models of MFi controllers may not be clickable. Let's dive into a code example to see how we can handle this.

Let's say we want to make the player character crouch when the player clicks L3. If we're supporting older versions of iOS, tvOS, or macOS, we should first check to see if we have the clickable thumbsticks API available to us.

Even if we do though, the specific controller we're mapping may not physically have clickable thumbsticks. So, we check to see if the left thumbstick button is new. And if it is, we should ensure that the user interaction we were intending to map to L3, in this case, crouching the player, is available through some other means.

Once we are working with a controller that we know has clickable thumbsticks, we can just assign our behavior to the press handler of L3. Now, when a player clicks the left thumbstick button, the game will toggle the player character's crouch.

If our game is running on versions of iOS, tvOS, or macOS that don't have this API, we still need to code an alternative path to the player crouch.

Next, let's talk about some new buttons that have been added to the API in support of the Xbox wireless and DualShock 4 Controllers, what we're calling the auxiliary buttons. These buttons are typically found in the middle of controllers.

First, we have the menu button. It should be used to pause gameplay and bring up the in-game pause menu. All supported controllers have this button.

Now let's look at the options button. It's a general purpose button, but you should avoid using it for time critical actions, as it's generally positioned out of the way. Note that not all controllers have this button.

Let's take a look at how these new APIs map to existing MFi controllers.

The center menu button maps directly to the button menu property of GCExtendedGamepad.

Note that in this example, this MFi controller doesn't have an options button. So, the button options property of GCExtendedGamepad is new.

Next up is the Xbox Wireless controller. When a controller has multiple auxiliary buttons, we determine its button mapping using positional equivalents.

The right auxiliary button, in this case, the Xbox Wireless controller's menu button maps directly to the button menu property, whereas the left Auxiliary button or the Xbox Wireless controller's view button maps to the button options property.

The Xbox button in the center of the controller has been reserved for system use.

The same approach can be applied for the DualShock 4.

Again, we can use positional equivalents to determine the button mappings.

The right auxiliary button or the DualShock 4's options button maps to the button menu property, whereas the left auxiliary button, the DualShock 4's share button maps to the button options property. The PS button in the center of the controller has also been reserved.

Note that we've deprecated the controller paused handler. You should now use the new button menu API instead. Let's dive into a code example to see how we can handle this.

Let's say we want to pause or unpause the game whenever the player presses the menu button. If we're supporting older versions of iOS, tvOS or macOS, we should first check to see if we have the new menu button API.

If we do, we simply register a pressed changed handler on the extended gamepad's button menu property.

Within the handler, we toggle the pause menu. Typically, when players want to pause the game, they want to do it as fast as possible, so we make sure to trigger this behavior on button press rather than button release.

If the new API isn't available, we should still register our functionality with the old controller PausedHandler. This will trigger whenever the menu button is pressed. So that was the menu button.

Now let's look at the options button. Again, remember that not all controllers may have this button. Let's dive into another code example to see how we should handle this.

Here, we want to bring up an in-game settings menu whenever the player presses the options button. If we're supporting older versions of iOS, tvOS, or macOS, we should first check to see if we have the new options button.

But even if we do, the controller we're mapping may not physically have an options button. So we check to see if the options button is new. And if it is, we need to ensure that the settings menu is accessible via some other means. For example, we can list it as an item in the pause menu.

If the controller has an options button, we can just assign our behavior to the button options pressed handler. Now, when a player clicks the options button, the game will toggle the settings menu.

If the new API isn't available, we still need to make sure there's an alternative path to reach the settings menu. So that was a quick look at the new API coming to the Game Controller framework this year in support of the newly added Xbox Wireless and DualShock 4 controllers.

Now, let's talk about some UI best practices for handling different controllers so that your users can experience your game in the best possible way.

With the Game Controller framework being expanded to support more than just MFi controllers, a new consideration arises. Buttons and other inputs may no longer be visually consistent between controllers. In this example, the B button in the prompt only matches the MFi controller while the Xbox Wireless controller's B button is red and the DualShock 4 has a circle button instead.

Let's dive deeper into this example to see how we could handle this. Let's say you're introducing your player to the base mechanics of your game and you want to display an in-game prompt that tells them how to block. In this case, your block function is mapped to the button B property of GCExtendedGamepad.

For an MFi controller, we display our MFi B button asset as we did before. For the Xbox Wireless controller, however, we need to slightly tweak the asset, in this case, by making the B button red.

Finally, we display a circle button when the DualShock 4 is the active controller. Now, regardless of what controller the player is using, they can easily understand your in-game prompts. Let's take a look at a code example to see how we can achieve this.

Here, we have a simple function called getBlockButtonAsset that returns the appropriate art asset for the block button.

First, we switch over the controller's product category, a new property added this year to GCController to determine what kind of controller it is. If it's an Xbox Wireless controller, we will return the red B button asset. If it's a DualShock 4 controller, we'll return the red circle button asset. And finally, by default, overturn the MFi green B button asset. This way, your game will support the diverse range of MFi controllers currently available and gracefully fall back to a sensible asset should new controller support be added in the future. Let's return to our previous block example. Instead of creating unique art assets for each type of controller, another option is to use generic symbols in your in-game tutorials and prompts.

By providing a generic prompt that doesn't rely on anything other than positional equivalents, you can rest assured knowing that your players will be able to easily understand your in-game guidance regardless of what type of controller is connected. In this example, it's clear that the players should press the right face button.

One last note, players may have multiple controllers connected to their device at the same time even if they're playing alone. You should gracefully support all connected controllers in this case, as a player may switch between them at any moment. This means that you'll need to recognize input from all connected controllers either by pulling them each frame or registering input handlers with each of them. You should also update your in-game guidance and visuals to match the most recently used controllers. For example, if a player is using their MFi controller, display the appropriate MFi art assets.

But if the player pressed an input on their connected DualShock 4, all in-game visuals should automatically adjust to use the DualShock 4 art assets. By dynamically adapting your visuals to the most recently used controller, you will provide your players an intuitive and polished controller-driven experience.

Now, for those of you who have previously implemented the game controller support outside of the Game Controller framework on macOS, we have some new guidance for your games.

When you adopt the Game Controller framework, game controllers will just work, so you should use it for MFi, Xbox Wireless, and DualShock 4 controller support on macOS. As we abstract away any hardware intricacies that may differ between controllers, we provide you consistency in your app across iOS, tvOS, and macOS. This will also help to ensure that your apps remain compatible with current and future OS updates. For those of you who have previously hardcoded support for these controllers through lower level APIs such as IOKit, we highly recommend that you move your implementation over to the Game Controller framework. As new controllers will appear in both IOKit and the Game Controller framework, you should drop your IOKit implementation to ensure there are no conflicts. This will allow you to code to a single interface and receive new game controller support for free.

So let's summarize what we've discussed in this video.

The Game Controller framework abstracts hardware through a common API, letting you write your code once without having to worry about the low-level differences between controller models. This has the great benefit of allowing your game to automatically support newly added controllers such as the Xbox Wireless and the DualShock 4 controllers. However, in order to provide the best user experience for your players, you should take care to adapt your game's UI and on-screen indicators to the active controller. And finally, ensure your macOS apps stay compatible with future OS updates by migrating existing controller support from lower level APIs to the Game Controller framework.

More information about this year's game controller update can be found on the Apple developer site.

That's it for this year's game controller's update. Thank you for watching. We hope you found this information helpful and you can use it to create some awesome games with game controller support.