Tutorial: Connect to Azure OpenAI Service in AKS using a connection string (preview)

In this tutorial, you learn how to create a pod in an Azure Kubernetes (AKS) cluster, which talks to Azure OpenAI Service using a connection string. You complete the following tasks:

Prerequisites

• An Azure account with an active subscription. Create an account for free.

• • Use the Bash environment in Azure Cloud Shell. For more information, see Quickstart for Bash in Azure Cloud Shell.

    

  • If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.

    • If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Sign in with the Azure CLI.

    • When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use extensions with the Azure CLI.

    • Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

• Docker and kubectl to manage container images and Kubernetes resources.
• A basic understanding of containers and AKS. Get started from preparing an application for AKS.
• Access permissions to create Azure OpenAI Service resources and deploy models.

Create Azure resources

1. Create a resource group for this tutorial.

   az group create \
       --name MyResourceGroup \
       --location eastus
   

2. Create an AKS cluster with the following command, or by referring to the AKS quickstart. This cluster is where we create the service connection and pod definition and deploy the sample application.

   az aks create \
       --resource-group MyResourceGroup \
       --name MyAKSCluster \
       --enable-managed-identity \
       --node-count 1 \
      --generate-ssh-keys
   

3. Connect to the cluster using the az aks get-credentials command.

   az aks get-credentials \
       --resource-group MyResourceGroup \
       --name MyAKSCluster
   

4. Create an Azure OpenAI Service resource using the az cognitiveservices account create command. Optionally refer to this tutorial for more instructions. Azure OpenAI Service is the target service that we'll connect to the AKS cluster.

   az cognitiveservices account create \
       --resource-group MyResourceGroup \
       --name MyOpenAIService \
       --location eastus \
       --kind OpenAI \
       --sku s0 \
       --custom-domain myopenaiservice \
       --subscription <SubscriptionID>
   

5. Deploy a model with the az cognitiveservices deployment create command. The model is used in the sample application to test the connection.

   az cognitiveservices account deployment create \
       --resource-group MyResourceGroup \
       --name MyOpenAIService
       --deployment-name MyModel \
       --model-name gpt-4 \
       --model-version 0613 \
       --model-format OpenAI \
       --sku-name "Standard"
       --capacity 1
   

6. Create an Azure Container Registry (ACR) resource with the az acr create command, or referring to this tutorial. The registry hosts the container image of the sample application, which the AKS pod definition consumes.

   az acr create \
       --resource-group MyResourceGroup \
       --name myregistry \
       --sku Standard
   

7. Enable anonymous pull using az acr update command so that the AKS cluster can consume the images in the registry.

   az acr update \
       --resource-group MyResourceGroup \
       --name myregistry \
       --anonymous-pull-enabled
   

Create a service connection in AKS with Service Connector (preview)

Create a service connection between an AKS cluster and Azure OpenAI Service in the Azure portal or the Azure CLI.

az aks connection create cognitiveservices --secret

Clone sample application

1. Clone the sample repository:

   git clone https://github.com/Azure-Samples/serviceconnector-aks-samples.git
   

2. Go to the repository's sample folder for Azure OpenAI:

   cd serviceconnector-aks-samples/azure-openai-connection-string
   

3. Replace the <MyModel> placeholder in the app.py file with the model name we deployed.

Build and push container images

1. Build and push the images to your container registry using the Azure CLI az acr build command.

   az acr build --registry <MyRegistry> --image sc-demo-openai-connstr:latest ./
   

2. View the images in your container registry using the az acr repository list command.

   az acr repository list --name <MyRegistry> --output table
   

Run application and test connection

1. Replace the placeholders in the pod.yaml file in the azure-openai-connection-string folder.

   • Replace <YourContainerImage> with the name of the image we built earlier. For example: <MyRegistry>.azurecr.io/sc-demo-openai-connstr:latest.
   • Replace <SecretCreatedByServiceConnector> with the secret created by Service Connector. You may check the secret name in the Azure portal, in the Service Connector pane.

2. Deploy the pod to your cluster with the kubectl apply command. Install kubectl locally using the az aks install-cli command if it isn't installed. The command creates a pod named sc-demo-openai-connstr in the default namespace of your AKS cluster.

   kubectl apply -f pod.yaml
   

3. Check if the deployment is successful by viewing the pod with kubectl.

   kubectl get pod/sc-demo-openai-connstr
   

4. Check that the connection is working by viewing the logs with kubectl.

   kubectl logs pod/sc-demo-openai-connstr
   

Clean up resources

If you don't need these resources anymore, clean up the Azure resources created in this tutorial by deleting the resource group.

az group delete \
    --resource-group MyResourceGroup

Next steps

Read the following articles to learn more about Service Connector concepts and how it helps AKS connect to Azure services.