Environment - Kubernetes resource

Azure DevOps Services | Azure DevOps Server 2022 | Azure DevOps Server 2020

The Kubernetes resource view shows the status of objects within the namespace that are mapped to the resource. The resource view also overlays pipeline traceability so you can trace back from a Kubernetes object to the pipeline, and then back to the commit.

Use Kubernetes resources to target Kubernetes clusters in an environment for deployment. Use pipelines to deploy to Azure Kubernetes Service (AKS) and clusters from any other cloud provider.

You can use Kubernetes resources with public or private clusters. For more information about how resources work, see resources in YAML and security with resources.

Note

If you're using a private AKS cluster, make sure you're connected to the cluster's virtual network as the the API server endpoint is not exposed through a public IP address.

Azure Pipelines recommends setting up a self-hosted agent within a VNET that has access to the cluster's virtual network. See Options for connecting to the private cluster for details.

Overview

See the following advantages of using Kubernetes resource views within environments:

  • Pipeline traceability - The Kubernetes manifest task, used for deployments, adds more annotations to show pipeline traceability in resource views. Pipeline traceability helps to identify the originating Azure DevOps organization, project, and pipeline responsible for updates that were made to an object within the namespace.

    Pipeline traceability

  • Diagnose resource health - Workload status can be useful for quickly debugging mistakes or regressions that were introduced by a new deployment. For example, for unconfigured imagePullSecrets resulting in ImagePullBackOff errors, pod status information can help you identify the root cause for the issue.

    ImagePullBackOff

  • Review App - Review App works by deploying every pull request from your Git repository to a dynamic Kubernetes resource under the environment. Reviewers can see how those changes look and work with other dependent services before they're merged into the target branch and deployed to production.

Use Azure Kubernetes Service

A ServiceAccount gets created in your chosen cluster and namespace when you use Azure Kubernetes Service (AKS). For a Kubernetes RBAC-enabled cluster, RoleBinding also gets created to limit the scope of the created service account to the chosen namespace. For a Kubernetes RBAC-disabled cluster, the ServiceAccount created has cluster-wide privileges (across namespaces).

Add an AKS Kubernetes resource

  1. In the environment details page, select Add resource and choose Kubernetes.

  2. Select Azure Kubernetes Service in the Provider dropdown.

  3. Choose the Azure subscription, cluster, and namespace (new/existing).

  4. Select Validate and create to create the Kubernetes resource.

  5. Verify that you see a cluster for your environment. You'll see the text "Never deployed" if you haven't yet deployed code to your cluster.

    Add a Kubernetes cluster.

Use an existing service account

The Azure Kubernetes Service maps a Kubernetes resource within your environment to a namespace.

For more information about setting up a Kubernetes service connection outside of an environment, see the Kubernetes service connection section in Service connections.

Tip

Use the generic provider (existing service account) to map a Kubernetes resource to a namespace from a non-AKS cluster.

Add a non-AKS Kubernetes resource

  1. In the environment details page, select Add resource and choose Kubernetes.

  2. Select Generic provider (existing service account) for your provider.

  3. Add the cluster name and namespace values.

  4. Add the server URL. You can get the URL with the following command:

    kubectl config view --minify -o 'jsonpath={.clusters[0].cluster.server}'
    
  5. To get the secret object.

    Kubernetes 1.22+

    Replace service-account-name with your account name.

    kubectl get secret -n <namespace>  -o jsonpath='{.items[?(@.metadata.annotations.kubernetes\.io/service-account\.name==\"service-account-name\")]}'
    

    If you get nothing, see Manually create a long-lived API token for a ServiceAccount.

    Kubernetes 1.22 and below:

    1. Find the service account secret name
    kubectl get serviceAccounts <service-account-name> -n <namespace> -o 'jsonpath={.secrets[*].name}'
    
    1. replace <service-account-secret-name> with the value in previous command in this command
    kubectl get secret <service-account-secret-name> -n <namespace> -o json
    
  6. Get the secret object using the output of the previous step.

    kubectl get secret <service-account-secret-name> -n <namespace> -o json
    
  7. Copy and paste the Secret object fetched in JSON form into the Secret field.

  8. Select Validate and create to create the Kubernetes resource.

Reference your Kubernetes resources in a pipeline

If you're using Azure Kubernetes Service and building a YAML pipeline, the easiest way to configure your pipeline is to use a template. Connect to your repository and select one of the following two Kubernetes Service options:

The templates let you set up Review App without needing to write YAML code from scratch or manually create explicit role bindings.

Kubernetes template options.

Set up Review App

In the following example, the first deployment job is run for non-PR branches and does deployments against a regular Kubernetes resource under environments. The second job runs only for PR branches and deploys against Review App resources (namespaces inside Kubernetes cluster) generated on demand. Resources get labeled with "Review" in the resource listing view of the environment. Define variables to use in the pipeline. If you use the Deploy to Azure Kubernetes Services template, these variables get defined for you.

# Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service
trigger:
- main

resources:
- repo: self

variables:

  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: '12345' # Docker service connection identifier
  envName: 'myEnv' # name of your environment
  imageRepository: 'name-of-image-repository' # name of image repository
  containerRegistry: 'mycontainer.azurecr.io' # path to container registry
  dockerfilePath: '**/Dockerfile'
  tag: '$(Build.BuildId)'
  imagePullSecret: 'my-app-secret' # image pull secret

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

  # Name of the new namespace being created to deploy the PR changes.
  k8sNamespaceForPR: 'review-app-$(System.PullRequest.PullRequestId)'

stages:
- stage: Build
  displayName: Build stage
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)

    - upload: manifests
      artifact: manifests

- stage: Production
  displayName: Deploy stage
  dependsOn: Build

  jobs:
  - deployment: Production
    condition: and(succeeded(), not(startsWith(variables['Build.SourceBranch'], 'refs/pull/')))
    displayName: Production
    pool:
      vmImage: $(vmImageName)
    environment: 
      name: $(envName).$(resourceName)
      resourceType: Kubernetes 
    strategy:
      runOnce:
        deploy:
          steps:
          - task: KubernetesManifest@0
            displayName: Create imagePullSecret
            inputs:
              action: createSecret
              secretName: $(imagePullSecret)
              dockerRegistryEndpoint: $(dockerRegistryServiceConnection)

          - task: KubernetesManifest@0
            displayName: Deploy to Kubernetes cluster
            inputs:
              action: deploy
              manifests: |
                $(Pipeline.Workspace)/manifests/deployment.yml
                $(Pipeline.Workspace)/manifests/service.yml
              imagePullSecrets: |
                $(imagePullSecret)
              containers: |
                $(containerRegistry)/$(imageRepository):$(tag)

  - deployment: DeployPullRequest
    displayName: Deploy Pull request
    condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/pull/'))
    pool:
      vmImage: $(vmImageName)

    environment: 
      name: $(envName).$(resourceName)
      resourceType: Kubernetes
    strategy:
      runOnce:
        deploy:
          steps:
          - reviewApp: default

          - task: Kubernetes@1
            displayName: 'Create a new namespace for the pull request'
            inputs:
              command: apply
              useConfigurationFile: true
              inline: '{ "kind": "Namespace", "apiVersion": "v1", "metadata": { "name": "$(k8sNamespaceForPR)" }}'

          - task: KubernetesManifest@0
            displayName: Create imagePullSecret
            inputs:
              action: createSecret
              secretName: $(imagePullSecret)
              namespace: $(k8sNamespaceForPR)
              dockerRegistryEndpoint: $(dockerRegistryServiceConnection)

          - task: KubernetesManifest@0
            displayName: Deploy to the new namespace in the Kubernetes cluster
            inputs:
              action: deploy
              namespace: $(k8sNamespaceForPR)
              manifests: |
                $(Pipeline.Workspace)/manifests/deployment.yml
                $(Pipeline.Workspace)/manifests/service.yml
              imagePullSecrets: |
                $(imagePullSecret)
              containers: |
                $(containerRegistry)/$(imageRepository):$(tag)

          - task: Kubernetes@1
            name: get
            displayName: 'Get services in the new namespace'
            continueOnError: true
            inputs:
              command: get
              namespace: $(k8sNamespaceForPR)
              arguments: svc
              outputFormat: jsonpath='http://{.items[0].status.loadBalancer.ingress[0].ip}:{.items[0].spec.ports[0].port}'

          # Getting the IP of the deployed service and writing it to a variable for posting comment
          - script: |
              url="$(get.KubectlOutput)"
              message="Your review app has been deployed"
              if [ ! -z "$url" -a "$url" != "http://:" ]
              then
                message="${message} and is available at $url.<br><br>[Learn More](https://aka.ms/testwithreviewapps) about how to test and provide feedback for the app."
              fi
              echo "##vso[task.setvariable variable=GITHUB_COMMENT]$message"

To use this job in an existing pipeline, the service connection backing the regular Kubernetes environment resource must be modified to "Use cluster admin credentials". Otherwise, role bindings must be created for the underlying service account to the Review App namespace.

Next steps