How many concurrent threads can be ran using one assistant.

Giridhar Vinukonda 40 Reputation points
2024-09-03T09:20:24.6233333+00:00

I have 100+ applications. Each application can have many user. I am considering to use one assistant for one application. I want to know the number of threads each assistant can use.

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,916 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. santoshkc 7,865 Reputation points Microsoft Vendor
    2024-09-03T15:47:11.2066667+00:00

    Hi @Giridhar Vinukonda,

    Thank you for reaching out to Microsoft Q&A forum!

    If you're using the Azure OpenAI Assistants platform, the number of concurrent threads that can be run using one assistant depends on the specific model and configuration. Azure OpenAI has certain limitations and quotas based on your subscription plan and the model's capacity.

    A chat session, also known as a thread within the Assistant's API, is where the conversation between the user and the assistant takes place. There is no limit to the number of messages in a thread, as the assistant automatically compresses requests to fit within the model's input token limit. Token management is fully abstracted and handled by the Assistant's API, meaning you don't control how many tokens are passed during each turn.

    For more info:

    I hope this helps. And, if you have any further query do let us know.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.