MicrosoftTokenizerLanguage Enum

Lists the languages supported by the Microsoft language tokenizer.

Inheritance
builtins.str
MicrosoftTokenizerLanguage
MicrosoftTokenizerLanguage

Constructor

MicrosoftTokenizerLanguage(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Fields

BANGLA
BULGARIAN
CATALAN
CHINESE_SIMPLIFIED
CHINESE_TRADITIONAL
CROATIAN
CZECH
DANISH
DUTCH
ENGLISH
FRENCH
GERMAN
GREEK
GUJARATI
HINDI
ICELANDIC
INDONESIAN
ITALIAN
JAPANESE
KANNADA
KOREAN
MALAY
MALAYALAM
MARATHI
NORWEGIAN_BOKMAAL
POLISH
PORTUGUESE
PORTUGUESE_BRAZILIAN
PUNJABI
ROMANIAN
RUSSIAN
SERBIAN_CYRILLIC
SERBIAN_LATIN
SLOVENIAN
SPANISH
SWEDISH
TAMIL
TELUGU
THAI
UKRAINIAN
URDU
VIETNAMESE