Recommendations for improving build velocity

  • Article

Applies to this Azure Well-Architected Framework Operational Excellence checklist recommendation:

OE:04 Optimize software development and quality assurance processes by following industry-proven practices for development and testing. For unambiguous role designation, standardize practices across components such as tooling, source control, application design patterns, documentation, and style guides.

Related guides: Recommendations for standardizing tools and processes | Recommendations for using continuous integration

This guide describes the recommendations for improving the performance of your deployment infrastructure. It's important to have a build process up and running the first day of your product development. Builds are the heartbeat of your continuous delivery system because the build status shows when your product is deployable. Builds provide crucial information about the status of your product, so you should always strive for fast builds.

It's difficult to fix a build problem if it takes longer to build. When delays happen and become normalized, teams tend to become less motivated to fix the problem.

Key design strategies

Optimize build times

To perform faster builds, you can:

  • Choose agents that meet your performance requirements: Speed up your builds by selecting the right build machines. Fast machines can make the difference between hours and minutes. If your pipelines are in Azure Pipelines, you can run your jobs by using a Microsoft-hosted agent. When you use Microsoft-hosted agents, maintenance and upgrades are taken care of for you. For more information, see Microsoft-hosted agents.

  • Optimize the build server location: When you're building your code, data is sent across the wire. Inputs to the builds are fetched from a source control repository and the artifact repository. The output from the build process needs to be copied, including the compiled artifacts, test reports, code coverage results, and debug symbols. It's important that these copy actions are run quickly. If you use your own build server, ensure that the build server is located near the sources and a target location. Fast uploads and downloads can reduce the overall build time.

  • Scale out build servers: A single build server might be sufficient for a small product. As the size and scope of the product and the number of teams working on the product increases, a single server might not be enough. Scale your infrastructure horizontally over multiple machines when you reach the limit. For more information, see Create and manage agent pools.

  • Optimize the build:

    • Add parallel jobs to speed up the build process. For more information, see Configure and pay for parallel jobs.

    • Enable parallel test suite runs, which often save a large amount of time, especially when running integration and UI tests. For more information, see Run tests in parallel for any test runner.

    • Use the notion of a multiplier, where you can scale out your builds over multiple build agents. For more information, see Specify jobs in your pipeline.

    • Consider moving integration, UI, and smoke tests to a release pipeline. Moving to a release pipeline improves the build speed and the speed of the build feedback loop.

    • Publish the build artifacts to a package management solution, such as NuGet or Maven. Publishing to a package management solution lets you reuse your build artifact more easily.

Minimize human intervention

Your organization might choose to create several different kinds of builds to optimize build times. Possible builds include:

  • Continuous integration (CI) build: The purpose of this build is to ensure code is compiled and unit tests are run. This build gets triggered at each commit. It serves as the heartbeat of the project and provides quality feedback to the team immediately. For more information, see Specify events that trigger pipelines.

  • Nightly build: The purpose of a nightly build isn't only to compile the code, but also to ensure any larger test suites that are inefficient run on a regular cadence for each build. Usually, these tests include integration, UI, or smoke tests. For more information, see Configure schedules for pipelines.

  • Release build: In addition to compiling and running tests, this build also compiles the API documentation, compliance reports, code signing, and other steps that aren't required every time the code is built. This build provides the golden copy that's pushed to the release pipeline to finally deploy in the production environment.

The types of builds needed by your organization depend on factors including your team's and organization's maturity, the kind of product you're working on, and your deployment strategy.

Azure facilitation

Azure DevOps is a collection of services that help you build a collaborative, efficient, and consistent development practice.

Use Azure Pipelines to build and release services to support continuous integration and continuous delivery (CI/CD) of your applications.

Use GitHub Actions for Azure to automate CI/CD processes and integrate directly with Azure to simplify deployments. You can also create workflows that build and test every pull request to your repository, or deploy merged pull requests to production by using GitHub Actions for Azure.

Microsoft-hosted agents are available natively in Azure Pipelines. These agents are single-use virtual machines that are only used for one job and then discarded, which provides an easy-to-manage option for your builds.

Operational Excellence checklist

Refer to the complete set of recommendations.

Operational Excellence checklist