ShingleTokenFilter Class

  • Reference

Creates combinations of tokens as a single token. This token filter is implemented using Apache Lucene.

All required parameters must be populated in order to send to server.

Inheritance
azure.search.documents.indexes._generated.models._models_py3.TokenFilter
ShingleTokenFilter

Constructor

ShingleTokenFilter(*, name: str, max_shingle_size: int = 2, min_shingle_size: int = 2, output_unigrams: bool = True, output_unigrams_if_no_shingles: bool = False, token_separator: str = ' ', filter_token: str = '_', **kwargs: Any)

Keyword-Only Parameters

Name Description
name
str

The name of the token filter. It must only contain letters, digits, spaces, dashes or underscores, can only start and end with alphanumeric characters, and is limited to 128 characters. Required.
max_shingle_size
int

The maximum shingle size. Default and minimum value is 2.

Default value: 2
min_shingle_size
int

The minimum shingle size. Default and minimum value is 2. Must be less than the value of maxShingleSize.

Default value: 2
output_unigrams
bool

A value indicating whether the output stream will contain the input tokens (unigrams) as well as shingles. Default is true.

Default value: True
output_unigrams_if_no_shingles
bool

A value indicating whether to output unigrams for those times when no shingles are available. This property takes precedence when outputUnigrams is set to false. Default is false.
token_separator
str

The string to use when joining adjacent tokens to form a shingle. Default is a single space (" ").

Default value:
filter_token
str

The string to insert for each position at which there is no token. Default is an underscore ("_").

Default value: _

Variables

Name Description
odata_type
str

A URI fragment specifying the type of token filter. Required.
name
str

The name of the token filter. It must only contain letters, digits, spaces, dashes or underscores, can only start and end with alphanumeric characters, and is limited to 128 characters. Required.
max_shingle_size
int

The maximum shingle size. Default and minimum value is 2.
min_shingle_size
int

The minimum shingle size. Default and minimum value is 2. Must be less than the value of maxShingleSize.
output_unigrams
bool

A value indicating whether the output stream will contain the input tokens (unigrams) as well as shingles. Default is true.
output_unigrams_if_no_shingles
bool

A value indicating whether to output unigrams for those times when no shingles are available. This property takes precedence when outputUnigrams is set to false. Default is false.
token_separator
str

The string to use when joining adjacent tokens to form a shingle. Default is a single space (" ").
filter_token
str

The string to insert for each position at which there is no token. Default is an underscore ("_").

Methods

as_dict

Return a dict that can be serialized using json.dump.

Advanced usage might optionally use a callback as parameter:

Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains 'type' with the msrest type and 'key' with the RestAPI encoded key. Value is the current value in this object.

The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.

See the three examples in this file:

  • attribute_transformer

  • full_restapi_key_transformer

  • last_restapi_key_transformer

If you want XML serialization, you can pass the kwargs is_xml=True.
deserialize

Parse a str using the RestAPI syntax and return a model.
enable_additional_properties_sending
from_dict

Parse a dict using given key extractor return a model.

By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
is_xml_model
serialize

Return the JSON that would be sent to server from this model.

This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).

If you want XML serialization, you can pass the kwargs is_xml=True.

as_dict

Return a dict that can be serialized using json.dump.

Advanced usage might optionally use a callback as parameter:

Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains 'type' with the msrest type and 'key' with the RestAPI encoded key. Value is the current value in this object.

The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.

See the three examples in this file:

  • attribute_transformer

  • full_restapi_key_transformer

  • last_restapi_key_transformer

If you want XML serialization, you can pass the kwargs is_xml=True.

as_dict(keep_readonly: bool = True, key_transformer: ~typing.Callable[[str, ~typing.Dict[str, ~typing.Any], ~typing.Any], ~typing.Any] = <function attribute_transformer>, **kwargs: ~typing.Any) -> MutableMapping[str, Any]

Parameters

Name Description
key_transformer
<xref:function>

A key transformer function.
keep_readonly
Default value: True

Returns

Type Description
dict

A dict JSON compatible object

deserialize

Parse a str using the RestAPI syntax and return a model.

deserialize(data: Any, content_type: str | None = None) -> ModelType

Parameters

Name Description
data
Required
str

A str using RestAPI structure. JSON by default.
content_type
str

JSON by default, set application/xml if XML.

Default value: None

Returns

Type Description

An instance of this model

Exceptions

Type Description
DeserializationError if something went wrong

enable_additional_properties_sending 

enable_additional_properties_sending() -> None

from_dict

Parse a dict using given key extractor return a model.

By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)

from_dict(data: Any, key_extractors: Callable[[str, Dict[str, Any], Any], Any] | None = None, content_type: str | None = None) -> ModelType

Parameters

Name Description
data
Required
dict

A dict using RestAPI structure
content_type
str

JSON by default, set application/xml if XML.

Default value: None
key_extractors
Default value: None

Returns

Type Description

An instance of this model

Exceptions

Type Description
DeserializationError if something went wrong

is_xml_model 

is_xml_model() -> bool

serialize

Return the JSON that would be sent to server from this model.

This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).

If you want XML serialization, you can pass the kwargs is_xml=True.

serialize(keep_readonly: bool = False, **kwargs: Any) -> MutableMapping[str, Any]

Parameters

Name Description
keep_readonly
bool

If you want to serialize the readonly attributes

Default value: False

Returns

Type Description
dict

A dict JSON compatible object