Small Basic: Character Set - Unicode

Introduction

This article explains about character set especially in Microsoft Small Basic programming language.

What is Character Set

A character set is an encoding code set for characters.  ASCII code was very popular as 7-bit character code.  In Small Basic, Unicode is used as it's character set.

What is Character Code

Character code is number for each character.  You can get any character code with Text.GetCharacterCode() operation.  Also you can get character from character code with Text.GetCharacter() operation. 

What is Unicode

Basically Unicode was created as 16-bit character code which includes many DBCS (double byte character set) codes for many countries.  In these days, many OS's, web sites, applications can use Unicode.  Usually Unicode is expressed such like U+0022.  This code means double quotation mark which code is 34 (0x22).  0x22 means hexadecimal.  A program Character Map In Windows Accessories can show Unicode for each character.  This tool shows only Unicode range between U+0000 and U+FFFF. 

And following table is from a program IME Pad which is included Japanese IME in Windows.  Range from U+0000 to U+FFFF is called BMP (Basic Multilingual Plane).  Range from U+10000 to U+1FFFF is called SMP (Supplementary Multilingual Plane).  Following table contains full BMP map and some part of SMP.  Other planes for range from U+20000 to U+10FFFF are not here. 

What is UTF-8

Small Basic source files and files created and read with File objects are encoded with UTF-8.  UTF-8 is stands for UCS (Universal Character Set) Transformation Format 8-bit.  UTF-8 is one of actual format to encode Unicode.  Alphabet is from U+0041 to U+007A.  These characters are single byte (8-bit) in ASCII code.  To reduce size for these popular characters, UTF-8 encoding allows characters between U+0000 and U+007F to be single bytes.  But you don't need to care about UTF-8 in Small Basic program.  Operations such as Text.GetLength, Text.GetSubText, Text.GetSubTextToEnd, and Text.GetIndexOf convert UTF-8 text to each Unicode character.

Characters for Game

Font Lucida Sans Unicode and some other fonts contain emoji characters that is suitable for game program.  Picture of Character Map above shows the characters. You can see them with following instructions.  You may find the other characters for your game.

1. Run Character Map.
2. Select Licida Sans Unicode font.
3. Check Advanced View check box.
4. Select Unicode Subrange in Group by.
5. Select Symbol & Dingbats for Unicode Subrange in Group by window.

And in SMP table above contains mahjong tiles, domino tiles, playing cards and so on.

Tips about Character Set

Glyph Depends on Font

Be careful about fonts.  Unicode has a lot of characters but many fonts don't have whole characters.  So some fonts show different glyph for the same character.  And some fonts are not installed in Windows system but with some applications that will be installed.  Small Basic program can be published.  So published program will be run in different environment, some has Office font, some doesn't and some has only Mac fonts.  See more detail about font here.

Character Set in TextWindow

And in TextWindow, the character set is not Unicode.  It depends on the localization, such like ASCII code in US, Shift-JIS code in Japan.

Characters in SMP

Text.ConvertToLowerCase(), Text.ConvertToUpperCase(), Text.GetCharacter(), Text.GetCharacterCode(), Text.GetIndexOf(), Text.GetLength(), Text.GetSubText() and Text.GetSubTextToEnd() operations don't support SMP (U+10000 to U+1FFFF) characters.  But SMP characters can be described in a literal.  See details in following SMP Characters sample.

Sample Code

ASCII Code Table

Program ID VQX212.  This program shows characters between U+0000 to U+007F.

Hexadecimal Dump

Program ID XWT217.  This program reads UTF-8 text file and simulate the UTF-8 encode of the file. Before running, remove automated comments of File objects.

Symbol Samples

Program ID QZS270.  This program shows symbols in Webdings,Wingdings fonts and Unicode.

Get Character from Unicode

Program ID RPZ143-2.  This program shows a character in a text box for a given Unicode.  So you can get (copy) the character.  This program found that SMP is not supported by Text objects in Small Basic.  Only the first 16-bit of the internal UTF-16 code is used.

SMP Characters

Program ID QBS151.  This program shows playing card characters in SMP range of Unicode using an array of literal .
Program ID FSQ891.  This program shows mahjong tile characters in SMP range of Unicode using an array of literal. 

See Also

• Small Basic: Text Basics
• Small Basic: Font
• Small Basic: Emoji

Additional Resources

• Full Emoji List | The Unicode Consortium

Other Languages

• Small Basic: Jeu de Caractères - Unicode (fr-FR)