Simulate Azure OpenAI API

Article
06/18/2024

When you build apps connected to Azure OpenAI, often, only a portion of the app interacts with the Azure OpenAI API. When you work on the portions of the app that don't require real replies from Azure OpenAI API, you can simulate the responses using Dev Proxy. Using simulated responses allows you to avoid incurring unnecessary costs. The OpenAIMockResponsePlugin uses a local language model running on Ollama to simulate responses from Azure OpenAI API.

Before you start

To simulate Azure OpenAI API responses using Dev Proxy, you need Ollama installed on your machine. To install Ollama, follow the instructions in the Ollama documentation.

By default, Dev Proxy uses the phi-3 language model. To use a different model, update the model property in the Dev Proxy configuration file.

Configure Dev Proxy to simulate Azure OpenAI API responses

Tip

Steps described in this tutorial are available in a ready-to-use Dev Proxy preset. To use the preset, in command line, run devproxy preset get simulate-azure-openai, and follow the instructions.

To simulate Azure OpenAI API responses using Dev Proxy, you need to enable the OpenAIMockResponsePlugin in the devproxyrc.json file.

{
  "$schema": "https://raw.githubusercontent.com/microsoft/dev-proxy/main/schemas/v0.19.0/rc.schema.json",
  "plugins": [
    {
      "name": "OpenAIMockResponsePlugin",
      "enabled": true,
      "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll"
    }
  ]
}

Next, configure Dev Proxy to intercept requests to Azure OpenAI API. For simplicity, use wildcards to intercept requests to all deployments.

{
  // [...] trimmed for brevity
  "urlsToWatch": [
    "https://*.openai.azure.com/openai/deployments/*/completions*"
  ]
}

Finally, configure Dev Proxy to use a local language model.

{
  // [...] trimmed for brevity
  "languageModel": {
    "enabled": true
  }
}

The complete configuration file looks like this.

{
  "$schema": "https://raw.githubusercontent.com/microsoft/dev-proxy/main/schemas/v0.19.0/rc.schema.json",
  "plugins": [
    {
      "name": "OpenAIMockResponsePlugin",
      "enabled": true,
      "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll"
    }
  ],
  "urlsToWatch": [
    "https://*.openai.azure.com/openai/deployments/*/completions*"
  ],
  "languageModel": {
    "enabled": true
  }
}