Why is the token usage event available in the stream from GPT4o-mini and not from GPT4-o

Hessel Wellema 226 Reputation points
2024-08-28T11:46:19.22+00:00

I noticed that an event.usage is available when answers are streamed from openai
This is great because we can now finally store the actual token usage per question

However: this feature only seams to be available in gpt4o-mini and not in gpt4-o

To enable it I use

            stream_options: {
                    include_usage: true
                },

In gpt4o-mini i get a nice event.usage object with promptTokens, completionTokens and totalTokens

In gpt4o however I get

Error during OpenAI API streaming: {message: "Unknown parameter: 'stream_options'.",type: 'invalid_request_error',param: 'stream_options',code: 'unknown_parameter'}

I don't want to use gpt4o-mini because I think it is crap.

Does anyone now if and when this will be available in GPT4o?

The api version (I tried 2024-05-13 and 2024-07-18) do not seam to have any effect on this

I am using

const { OpenAIClient } = require('@azure/openai');

        "@azure/openai": "^1.0.0-beta.11",

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,916 questions
{count} vote

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.