AzureOpenAITokenizerParameters Class

  • Reference

Definition

Namespace:
Azure.Search.Documents.Indexes.Models
Assembly:
Azure.Search.Documents.dll
Package:
Azure.Search.Documents v11.7.0-beta.1
Source:
AzureOpenAITokenizerParameters.cs

The AzureOpenAITokenizerParameters.

public class AzureOpenAITokenizerParameters
type AzureOpenAITokenizerParameters = class
Public Class AzureOpenAITokenizerParameters
Inheritance
Object
AzureOpenAITokenizerParameters

Constructors

AzureOpenAITokenizerParameters()

Initializes a new instance of AzureOpenAITokenizerParameters.

Properties

AllowedSpecialTokens

(Optional) Only applies if the unit is set to azureOpenAITokens. This parameter defines a collection of special tokens that are permitted within the tokenization process.
EncoderModelName

Only applies if the unit is set to azureOpenAITokens. Options include 'R50k_base', 'P50k_base', 'P50k_edit' and 'CL100k_base'. The default value is 'CL100k_base'.

Applies to