StandardTokenizer.MaxTokenLength Property

Definition

Gets or sets the maximum token length. Default is 255. Tokens longer than the maximum length are split

[Newtonsoft.Json.JsonProperty(PropertyName="maxTokenLength")]
public int? MaxTokenLength { get; set; }
[<Newtonsoft.Json.JsonProperty(PropertyName="maxTokenLength")>]
member this.MaxTokenLength : Nullable<int> with get, set
Public Property MaxTokenLength As Nullable(Of Integer)

Property Value

Attributes
Newtonsoft.Json.JsonPropertyAttribute

Applies to