Extend your Xcode Cloud workflows

Hi I’m Daniel, and later I’ll be joined by my colleague Colin. Today I’m really excited to talk about some powerful and extensible features in Xcode Cloud that allow you to scale and extend your workflows. Xcode Cloud is a continuous integration and delivery service built into Xcode and also available in App Store Connect. It’s designed expressly for Apple developers and accelerates the development and delivery of high-quality apps by bringing together cloud-based tools that help you build apps, run automated tests in parallel, deliver apps to testers, and view and manage user feedback which helps you build better apps.

In this session, we’re going to explore how you can extend your Xcode Cloud workflows. We’ll start by reviewing some essential workflow concepts. Next, we’ll show you how to scale your workflows as your app grows. And finally, we’ll extend a workflow to work with systems beyond Xcode Cloud. Let’s get started by reviewing some essential workflow concepts. All Apple Developer Program memberships come with 25 hours of monthly build time. By the end of this session, you’ll have the tools necessary to get started for the first time or make your workflows more flexible.

As we’ve just covered, there’s a ton Xcode Cloud can do you for and your development but it all starts with a simple workflow. A workflow is made up of a four elements. The Environment, Start Conditions, Build Actions, and Post Actions. The Environment is where you define Environment Variables for your workflow. It’s also where you decide which Xcode and macOS version your workflow will use when it runs. You can pick a specific version or choose from a set of helpful aliases like latest release. Start Conditions define when a workflow runs. You can choose to respond to source control events like when a Branch, Pull Request, or git Tag updates. You can also choose to set up a schedule so that your workflow runs at a specific day and time. Or you can opt for a manual start condition which will only run manually. Build actions describe what you want Xcode Cloud to do with your source code and a workflow can be configured with one or more of them. You can choose to Build your app, run Tests, Analyze, or Archive your app to prepare it for distribution. Finally, Post Actions define what happens after your build actions have completed running. You may want to notify your team over Slack when your build is completed or do something with an archived app like notarize it or distribute to TestFlight. For most cases, a simple workflow is all that’s needed to get a lot done. But workflows are powerful and can be scaled to meet your needs as your app grows. We recently added some functionality to an app we’re working on so that it’s data comes from a service. We have an existing Xcode Cloud workflow which starts when code is pushed to our main branch. It builds our latest changes then it runs our tests, and finally, changes are delivered to our testers on TestFlight. Our test suite is a mix of unit and UI tests that rely on mocked data and this is great for exercising UI and logic our app owns. But now that our app has a dependency on a service we’ve added a few integration tests that talk to a test server. Integration tests are a useful way to ensure an app works end to end with its dependencies. However in our case, they take more time and resources to run than other tests and can be affected by issues our app can’t control like network conditions. We want to be able to configure a workflow that runs our integration tests, but because they depend on a server and make real network requests, it would be great to have more control over when this workflow runs. New in Xcode 15.1, you can configure workflows to be started manually with manual start conditions. Let’s look at how to do this now. Let’s start by creating a new workflow that only runs my new test plan. To save some time, I’m going to duplicate my existing workflow. I can see my workflows by secondary clicking on my app in the Cloud reports navigator and selecting Manage Workflows. From the Manage Workflows view, I can secondary click on my existing workflow and select the Duplicate action. This opens a new workflow in the workflow editor. I’m going to rename the workflow to Integration Tests and leave the Description blank. Next, I’ll add a Manual Start condition and remove the existing one.

This workflow can now only be started when I tell it to and not from any other events. I’ll accept the default option to associate the start condition with any branch but I could also specify a branch, PR or git tag to be more specific about what git references this start condition can associate with. I'll change the Test action to run my IntegrationTests test plan.

And for now, since nothing should be done when this workflow finishes, I’m going to also remove the Post Actions and click Save.

You can of course configure workflows to run using different versions of Xcode and macOS specified in the Environment tab. However, in our case we want to ensure that both of our workflows always run using the same versions. Custom aliases are new in Xcode 15.3. Xcode Cloud already provides some aliases you may have seen like latest release which ensures that your workflows will run on the latest versions we have made available. And now you can define your own aliases for your team. When you use your alias in one or more workflows, those workflows will run using the version of Xcode or macOS specified in the alias. That means when you update your alias, all of the workflows using it will run with the updated values. Let’s use this feature with our new workflow we just created. I can see all of my Custom Aliases for a given app by secondary clicking the app in the navigator and selecting Manage Custom Aliases.

I’ll click on the plus button and create a new Xcode alias. This will bring up the Custom Alias editor. Here I can give my Custom Alias a name and select the version of Xcode it resolves to. In this case I’m going to call it Team Preference and select Xcode 15.3.

I click save and I’ll do the same thing for a macOS alias.

To adopt the alias into my workflow, I’ll navigate back to the manage workflows view and open my Integration Tests workflow to its Environment tab.

In the Xcode version drop down, I can now see my custom alias. I’ll select it and do the same thing for macOS.

You can also quickly create and manage Custom Aliases from the version selector drop down menu here or from the Integrations menu item. Finally, I’ll go back to my first workflow, and update the Xcode and macOS versions to use my new alias.

Perfect! Now whenever I update to a new version in my aliases, both workflows will get that change. You can learn more about how to use custom aliases in the documentation. Our integration tests will work great in Xcode Cloud with no changes, but we can make them a little smarter by only running them if our test server is online. Let’s take advantage of another great feature of Xcode Cloud to do this by adding a custom script to our project. You can define custom scripts inside of your repository that will be run at specific points in your build. Either after your repository is cloned, before xcodebuild is run, or after xcodebuild is run.

For more information on custom scripts, check out "Customize your advanced Xcode Cloud workflows".

All environment variables defined in your workflow, as well as environment variables provided by Xcode Cloud are available to use in these scripts. Today we’ll use two of them in our custom script. To see the full list of environment variables you can use, head over to the “Environment variable reference” page in the documentation.

Xcode Cloud expects all custom scripts to be in a folder called ci_scripts in the root of your project. The script’s file name determines the point in the build when it will be executed.

In this scenario, we want the script to execute right before our tests run.

In Xcode, I’ll secondary click on my project in the project navigator to create the scripts folder.

I’ll add a new script file to my folder and give it the name ci_pre_xcodebuild.sh so that it runs before our tests.

I only want to take action for my integration tests, so I’ll add some logic to check environment variables for the build action and workflowID.

Here the script checks that we’re running for the test build action and that our workflow matches an identifier. To get the ID of my workflow, I can simply navigate to the Cloud reports navigator, secondary click on the workflow and select Copy Workflow ID.

Now that I have that, I’ll go back to my script and paste it in.

Inside of this block, I’m going to use curl to call our server’s health check endpoint, and print out detailed error logs on failure.

By specifying the set -e shell option, this script will exit immediately if it encounters an error and our workflow will stop there which is exactly what we want if our server is unreachable. Note that because Xcode Cloud builds run on ephemeral task workers, the range of IP addresses that the host uses will vary. I’ve already ensured that Xcode Cloud can talk to my server by adding the required IP address ranges to my server’s firewall’s inbound allowlist. To get the specifics on which IP address to allow, please refer to the “Requirements for using Xcode Cloud” documentation page. We just used Manual Start conditions to build a new workflow for our integration tests and used Custom Aliases to ensure our workflows keep their macOS and Xcode versions in sync. We also leveraged the power of custom scripts to make our tests a little smarter and fail earlier in the event of an unreachable test server. To explore the ways Xcode Cloud can connect with other systems and take workflows even further, I’ll hand it over to Colin. Thanks Daniel! Now that we’re familiar with workflows and how they can be scaled, let’s build off of the work we’ve done so far and see how they can connect with systems outside of Xcode Cloud. As Daniel mentioned earlier, our app now depends on a service. With this new dependency comes additional risk, since someone could push a code change to the server which breaks our app. We can alleviate some of this risk by running our new Integration Tests workflow. This will generate test results which tell us if our app is compatible with the latest server changes. Right now, the workflow only runs manually, but we can use the App Store Connect API to automatically start a build whenever our test server has new changes.

The App Store Connect API enables you to automate a wide range of App Store Connect tasks; including those related to Xcode Cloud. You can learn more about these endpoints and how to call them, in the documentation. Let’s work backwards to see what’s needed to run our Integration Tests workflow using the App Store Connect API. The CiBuildRuns endpoint lets us create new Xcode Cloud builds. When calling it, we must specify identifiers for the workflow we want to run, and the gitReference we want to build. We know the ID of our Integration Tests workflow, but don’t know the gitReference of the branch we want to build. We can find this by using the ScmRepositories endpoint. We can call this endpoint using a repositoryID to fetch all of the branches, tags, and pull requests associated with that repo. But first, we need to know what repositoryID our workflow is using. This is available as part of the CiWorkflows resource, which can be queried using our workflowID. In total, our script must make three API calls to create the build we want.

The App Store Connect API is specified using OpenAPI so we can use a code generator to create strongly typed Swift code for each endpoint. In this demo we’ll focus on the implementation, but you can learn more about the open source code generator I’m using in "Meet Swift OpenAPI Generator". Since we need to make three API calls to create our build, I'll write some extensions on the generated API Client for convenience. We need a function that takes a workflowID as a parameter, fetches it’s associated CiWorkflows resource, then returns the repositoryID.

Next, we need a way to translate that repositoryID into the gitReferenceID that we want to build. We can do this using the SCMRepositories endpoint by fetching all of the gitReferences associated with our repository, and returning the gitReferenceID for the branch with a specified name.

Finally, we want to start a new build. To do that, we'll need the workflowID and gitReferenceID, which we can pass to the ciBuildRuns endpoint. Now let’s put everything together. First, we’ll call the repoID function with the workflowID for our Integration Tests workflow.

Next, we’ll call branchID with the name of the branch we want to build. In this case, I’ll use “main”.

Finally, we’ll use the workflowID, and gitReferenceID to call startBuild, completing our script.

Note that builds started by the App Store Connect API are considered manual, therefore you should ensure that you have a manual start condition setup that matches the reference you’re building. In our case, Daniel already setup this workflow with a manual start condition that matches any branch, so we won’t have issues. With our script complete, we can now generate test results whenever new server code is pushed. We could stop here, but we probably want to do something with the test results.

We want to connect the work that we’ve done so far to a Production environment, so that the server changes get deployed after being validated in the Test environment. We could check the test results by hand and manually deploy to production as needed, but we can do better. Instead, we can utilize the Xcode Cloud webhooks feature to connect our test results to a deployment service, and then push the server changes to production automatically when they pass.

Webhooks allow services to respond to build events as they occur, so you can integrate Xcode Cloud into other services and tools that your development process depends on. You can learn more about webhooks and how to configure them in the documentation. All you need to listen and respond to webhooks is an HTTP server. You can setup a simple project using Swift on Server in just a few steps using either the Vapor or Hummingbird frameworks. For this demo, I want to focus on the implementation, so I’ve already setup a Vapor server with an endpoint that can receive webhooks. When you configure a webhook, Xcode Cloud will send requests to your endpoint containing a JSON payload with detailed information about different build events. Here, I’ve defined a WebhookPayload struct containing the fields that we’re interested in. For our scenario, we need the workflowID, buildID, build execution progress, and build completion state.

I’ll use this struct to decode the webhook request so I can easily extract fields out of the payload.

Next, I’ll add some logic to compare the workflowID from the payload to the workflowID of our Integration Tests workflow.

There are two similar fields which describe the state of the build event. ExecutionProgress describes whether the build is running or has completed, while completionStatus describes whether the build succeeded or failed. I’ll update my if statement with checks for both, to ensure that the build completed successfully.

In the body of this statement, I’ll tell a deployment service that the Integration Tests have passed, and send the buildID. Our deployment service can use this information, along with other checks to approve and deploy the server changes to production.

The last thing we need to do is tell Xcode Cloud to send build events to our webhook. We can do this from the Xcode Cloud view in App Store Connect.

With my app selected, I’ll go to Settings and click the Webhooks tab.

Clicking the plus button lets me configure a new web hook. I’ll give it a name and add the URL of my listener.

With our web hook setup, we have all of the pieces in place to connect our server changes through Xcode Cloud. Let’s review the work that we’ve done so far.

Whenever a new code change is deployed to our service’s test environment, we’ll call the App Store Connect API to start a build of our Integration Tests workflow. This runs tests validating the integration between our app and test server. The results are processed by our webhook listener and if all the tests pass, then the changes are deployed to production.

In the case where everything passes, our pipeline is fully automated from code push to production deployment. But in the case where a server change is pushed which causes issues in our app. The integration tests will fail and our webhook listener will prevent the change from being deployed to production; which is exactly what we want.

Since we’re talking about a server code change, the team member who pushed the change may not even have Xcode installed on their machine. But because Xcode Cloud is integrated inside of App Store Connect, they can still view and investigate issues from their browser. Here are the build results from App Store Connect of a run where things didn’t go as expected. From the Actions section, it’s clear that the was an issue with our IntegrationTests action. Clicking on the Tests section will open up the test report.

Even though I’m in App Store Connect, I can see and do everything as I would expect to in Xcode. I can filter by test status, search by test suite, and view detailed information about each test. In this case, I can see that there was a problem with the data returned from the server. If I needed to investigate further, I could click on Artifacts in the sidebar to download a variety of files related to the build like Xcode logs, and crash reports. We’ve covered a lot in this session, but these are just a few of the ways Xcode Cloud workflows can be extended.

We used manual start conditions, custom aliases, and custom scripts to scale our team’s workflows; and integrated them into other systems using the App Store Connect API and webhooks. Check out our other sessions like "Create practical workflows in Xcode Cloud" and "Meet Xcode Cloud" to learn more about the ways Xcode Cloud can enhance your team’s CI process. We hope you’re walking away with some new ideas of how these concepts can be applied to your own workflows. Thanks for watching!