Does gpt-4-1106-preview support logprobs?

Rohan Pathak 15 Reputation points
2024-06-05T18:14:56.17+00:00

I am using the openai Python package to import AzureOpenAI, and I have created a client for my gpt-4 deployment (1106-preview) with API version 2024-02-01. I am using Azure AI Search to perform RAG using the extra_body parameter, with a data source pointing to my vector database.

I would like to generate some confidence scores for all of the LLM responses, ideally using logprobs = True when using client.chat.completions.create. However, I keep receiving errors saying that log_probs is not supported, nor is logprobs, despite the documentation showing here: https://video2.skills-academy.com/en-us/azure/ai-services/openai/reference#chat-completions that it can be used.

Is this unsupported? Is there additional configuration for me to add prior to including logprobs=True in my client.chat.completions.create request?

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,536 questions
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,577 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
236 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. AshokPeddakotla-MSFT 30,066 Reputation points
    2024-06-12T12:28:37.44+00:00

    Rohan Pathak I have checked internally and this parameter is not available in the new models, including gpt-4 and gpt-4-32k. Our team is actively working on supporting logprobs.

    Also, there is no exact ETA which can be shared now.

    Kindly keep an eye on the What's new in Azure OpenAI Service page for more updates.

    Thanks for your understanding. Do let me know if you have any further queries.

    0 comments