Annotations for Application Gateway Ingress Controller

The Kubernetes Ingress resource can be annotated with arbitrary key/value pairs. AGIC relies on annotations to program Application Gateway features, which aren't configurable using the Ingress YAML. Ingress annotations are applied to all HTTP settings, backend pools, and listeners derived from an ingress resource.

List of supported annotations

For an Ingress resource to be observed by AGIC, it must be annotated with kubernetes.io/ingress.class: azure/application-gateway. Only then AGIC works with the Ingress resource in question.

Backend Path Prefix

The following annotation allows the backend path specified in an ingress resource to be rewritten with prefix specified in this annotation. It allows users to expose services whose endpoints are different than endpoint names used to expose a service in an ingress resource.

Usage

appgw.ingress.kubernetes.io/backend-path-prefix: <path prefix>

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-bkprefix
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-path-prefix: "/test/"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

In the previous example, you've defined an ingress resource named go-server-ingress-bkprefix with an annotation appgw.ingress.kubernetes.io/backend-path-prefix: "/test/". The annotation tells application gateway to create an HTTP setting, which has a path prefix override for the path /hello to /test/.

Backend Hostname

This annotation allows us to specify the host name that Application Gateway should use while talking to the Pods.

Usage

appgw.ingress.kubernetes.io/backend-hostname: "internal.example.com"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-timeout
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-hostname: "internal.example.com"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        backend:
          service:
            name: store-service
            port:
              number: 80
        pathType: Exact

Custom Health Probe

Application Gateway can be configured to send custom health probes to the backend address pool. When these annotations are present, Kubernetes Ingress controller creates a custom probe to monitor the backend application and applies the changes to the application gateway.

health-probe-hostname: This annotation allows a custom hostname on the health probe.
health-probe-port: This annotation configures a custom health probe port.
health-probe-path: This annotation defines a path for the health probe.
health-probe-status-code: This annotation allows the health probe to accept different HTTP status codes.
health-probe-interval: This annotation defines the interval that the health probe runs at.
health-probe-timeout: This annotation defines how long the health probe will wait for a response before failing the probe.
health-probe-unhealthy-threshold: This annotation defines how many health probes must fail for the backend to be marked as unhealthy.

Usage

appgw.ingress.kubernetes.io/health-probe-hostname: "contoso.com"
appgw.ingress.kubernetes.io/health-probe-port: 80
appgw.ingress.kubernetes.io/health-probe-path: "/"
appgw.ingress.kubernetes.io/health-probe-status-code: "100-599"
appgw.ingress.kubernetes.io/health-probe-interval: 30
appgw.ingress.kubernetes.io/health-probe-timeout: 30
appgw.ingress.kubernetes.io/health-probe-unhealthy-threshold: 2

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/health-probe-hostname: "contoso.com"
    appgw.ingress.kubernetes.io/health-probe-port: 81
    appgw.ingress.kubernetes.io/health-probe-path: "/probepath"
    appgw.ingress.kubernetes.io/health-probe-status-code: "100-599"
    appgw.ingress.kubernetes.io/health-probe-interval: 31
    appgw.ingress.kubernetes.io/health-probe-timeout: 31
    appgw.ingress.kubernetes.io/health-probe-unhealthy-threshold: 2
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

TLS Redirect

Application Gateway can be configured to automatically redirect HTTP URLs to their HTTPS counterparts. When this annotation is present and TLS is properly configured, Kubernetes Ingress controller creates a routing rule with a redirection configuration and applies the changes to your Application Gateway. The redirect created will be HTTP 301 Moved Permanently.

Usage

appgw.ingress.kubernetes.io/ssl-redirect: "true"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-redirect
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
   - hosts:
     - www.contoso.com
     secretName: testsecret-tls
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Connection Draining

connection-draining: This annotation allows us to specify whether to enable connection draining. connection-draining-timeout: This annotation allows us to specify a timeout, after which Application Gateway terminates the requests to the draining backend endpoint.

Usage

appgw.ingress.kubernetes.io/connection-draining: "true"
appgw.ingress.kubernetes.io/connection-draining-timeout: "60"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-drain
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/connection-draining: "true"
    appgw.ingress.kubernetes.io/connection-draining-timeout: "60"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Cookie Based Affinity

The following annotation allows you to specify whether to enable cookie based affinity.

Usage

appgw.ingress.kubernetes.io/cookie-based-affinity: "true"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-affinity
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/cookie-based-affinity: "true"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Request Timeout

The following annotation allows you to specify the request timeout in seconds, after which Application Gateway fails the request if response is not received.

Usage

appgw.ingress.kubernetes.io/request-timeout: "20"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-timeout
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/request-timeout: "20"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Use Private IP

The following annotation allows you to specify whether to expose this endpoint on Private IP of Application Gateway.

Usage

appgw.ingress.kubernetes.io/use-private-ip: "true"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-privateip
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/use-private-ip: "true"
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Override Frontend Port

The annotation allows you to configure a frontend listener to use different ports other than 80/443 for http/https.

If the port is within the App Gw authorized range (1 - 64999), this listener will be created on this specific port. If an invalid port or no port is set in the annotation, the configuration will fall back on default 80 or 443.

Usage

appgw.ingress.kubernetes.io/override-frontend-port: "port"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-overridefrontendport
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/override-frontend-port: "8080"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        backend:
          service:
            name: store-service
            port:
              number: 80
        pathType: Exact

Backend Protocol

The following annotation allows you to specify the protocol that Application Gateway should use while communicating with the pods. Supported Protocols are http and https.

Usage

appgw.ingress.kubernetes.io/backend-protocol: "https"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-timeout
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-protocol: "https"
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 443

Hostname Extension

Application Gateway can be configured to accept multiple hostnames. The hostname-extention annotation allows for this by letting you define multiple hostnames including wildcard hostnames. This will append the hostnames onto the FQDN that is defined in the ingress spec.rules.host on the frontend listener so it is configured as a multisite listener.

Usage

appgw.ingress.kubernetes.io/hostname-extension: "hostname1, hostname2"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-multisite
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/hostname-extension: "hostname1, hostname2"
spec:
  rules:
  - host: contoso.com
    http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 443

WAF Policy for Path

This annotation allows you to attach an already created WAF policy to the list paths for a host within a Kubernetes Ingress resource being annotated.

Usage

appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/SampleRG/providers/Microsoft.Network/applicationGatewayWebApplicationFirewallPolicies/AGICWAFPolcy"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ad-server-ingress
  namespace: commerce
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/abcd/resourceGroups/rg/providers/Microsoft.Network/applicationGatewayWebApplicationFirewallPolicies/adserver"
spec:
  rules:
  - http:
      paths:
      - path: /ad-server
        backend:
          service:
            name: ad-server
            port:
              number: 80
        pathType: Exact
      - path: /auth
        backend:
          service:
            name: auth-server
            port:
              number: 80
        pathType: Exact

Application Gateway SSL Certificate

The SSL certificate can be configured to Application Gateway either from a local PFX certificate file or a reference to an Azure Key Vault unversioned secret ID. When the annotation is present with a certificate name and the certificate is pre-installed in Application Gateway, Kubernetes Ingress controller will create a routing rule with a HTTPS listener and apply the changes to your App Gateway. appgw-ssl-certificate annotation can also be used together with ssl-redirect annotation in case of SSL redirect.

Please refer to appgw-ssl-certificate feature for more details.

Usage

appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-certificate
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"
spec:
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Application Gateway SSL Profile

Users can configure a ssl profile on the Application Gateway per listener. When the annotation is present with a profile name and the profile is pre-installed in the Application Gateway, Kubernetes Ingress controller will create a routing rule with a HTTPS listener and apply the changes to your App Gateway.

Usage

appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"
appgw.ingress.kubernetes.io/appgw-ssl-profile: "SampleSSLProfile"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-certificate
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"
    appgw.ingress.kubernetes.io/appgw-ssl-profile: "SampleSSLProfile"
spec:
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Application Gateway Trusted Root Certificate

Users now can configure their own root certificates to Application Gateway to be trusted via AGIC. The annotation appgw-trusted-root-certificate can be used together with annotation backend-protocol to indicate end-to-end ssl encryption, multiple root certificates, separated by comma, if specified, for example, "name-of-my-root-cert1,name-of-my-root-certificate2".

Usage

appgw.ingress.kubernetes.io/backend-protocol: "https"
appgw.ingress.kubernetes.io/appgw-trusted-root-certificate: "name-of-my-root-cert1"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-certificate
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-protocol: "https"
    appgw.ingress.kubernetes.io/appgw-trusted-root-certificate: "name-of-my-root-cert1"
spec:
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Rewrite Rule Set

The following annotation allows you to assign an existing rewrite rule set to the corresponding request routing rule.

Usage

appgw.ingress.kubernetes.io/rewrite-rule-set: <rewrite rule set name>

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-bkprefix
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/rewrite-rule-set: add-custom-response-header
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 8080

Rewrite Rule Set Custom Resource

Application Gateway allows you to rewrite selected content of requests and responses. With this feature, you can translate URLs, query string parameters as well as modify request and response headers. It also allows you to add conditions to ensure that the URL or the specified headers are rewritten only when certain conditions are met. These conditions are based on the request and response information. Rewrite Rule Set Custom Resource brings this feature to AGIC.

HTTP headers allow a client and server to pass additional information with a request or response. By rewriting these headers, you can accomplish important tasks, such as adding security-related header fields like HSTS/ X-XSS-Protection, removing response header fields that might reveal sensitive information, and removing port information from X-Forwarded-For headers.

With URL rewrite capability, you can: - Rewrite the host name, path and query string of the request URL - Choose to rewrite the URL of all requests or only those requests which match one or more of the conditions you set. These conditions are based on the request and response properties (request header, response header and server variables). - Choose to route the request based on either the original URL or the rewritten URL

Usage

appgw.ingress.kubernetes.io/rewrite-rule-set-custom-resource

Example

apiVersion: appgw.ingress.azure.io/v1beta1 
kind: AzureApplicationGatewayRewrite 
metadata: 
  name: my-rewrite-rule-set-custom-resource 
spec: 
  rewriteRules: 
  - name: rule1 
    ruleSequence: 21
    conditions:
  - ignoreCase: false
    negate: false
    variable: http_req_Host
    pattern: example.com
  actions:
    requestHeaderConfigurations:
    - actionType: set
      headerName: incoming-test-header
      headerValue: incoming-test-value
    responseHeaderConfigurations:
    - actionType: set
      headerName: outgoing-test-header
      headerValue: outgoing-test-value
    urlConfiguration:
      modifiedPath: "/api/"
      modifiedQueryString: "query=test-value"
      reroute: false

Rule Priority

This annotation allows for application gateway ingress controller to explicitly set the priority of the associated Request Routing Rules.

Usage

appgw.ingress.kubernetes.io/rule-priority:

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-rulepriority
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/rule-priority: 10
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 8080

In the above example the request routing rule would have a priority of 10 set.