Semiconductor & Computer Engineering
This page does not attempt to describe what is Unicode but rather how it works around mIRC. The technical terms are omitted on purpose.
Before Unicode (mIRC 7.x), mIRC supported code pages, handling various language used in the world, code pages are encoding on 8 bits (byte), 8 bits can be used to represent 256 values, mIRC also supported others encoding for Japanese system for example. Basically, code pages are all based on ASCII, which defines 128 characters, assigned to the first 128 values represented in 8 bits, and then each code page adds the required characters for the language, é for the french or a Greek letter for Greek people.
You can see Unicode as a new codepage, but which defines 1,114,112 characters, including all languages. This is much better for IRC, which can be used from all over the world.
They are different ways to implement the handling of Unicode in a application, mIRC uses the utf16 encoding internally, before it was handled using US-ASCII.
Two mains reasons for this:
- The most frequently used character, the first 65635 ones, can be stored with 16 bits, as a result, routine dealing with Unicode are going to be faster.
- It uses less memory than utf32.
Drawback: each character in a script is depicted as a 16 bits unit, $asc and $chr cannot be used with characters over 65635, but you can form those others characters in your script by combining two 16 bits characters together:
//var -s %a = $chr(55384), %b = $chr(56320) | echo -a %a $+ %b 𦀀
You can also express those characters with their utf8 representation using $utfdecode, which decode utf8:
;you can use utf8 to form the character 65536 for example, which is four bytes in utf8 (f0 90 80 80): //var -s %a $utfdecode($chr($base(f0,16,10)) $+ $chr($base(90,16,10)) $+ $chr($base(80,16,10)) $+ $chr($base(80,16,10))) 𐀀
$utfencode can be used to encode text to utf8:
//var -s %a $utfencode(é) é
The scripting language still somewhat support code pages, you can decode text to utf8 while the bytes in the text are interpreted in the given code page.
Each code page has a number (Gdi charset):
- 000 - ANSI_CHARSET
- 001 - DEFAULT_CHARSET
- 002 - SYMBOL_CHARSET
- 077 - MAC_CHARSET
- 128 - SHIFTJIS_CHARSET
- 129 - HANGEUL_CHARSET
- 130 - JOHAB_CHARSET
- 134 - GB2312_CHARSET
- 136 - CHINESEBIG5_CHARSET
- 161 - GREEK_CHARSET
- 162 - TURKISH_CHARSET
- 163 - VIETNAMESE_CHARSET
- 177 - HEBREW_CHARSET
- 178 - ARABIC_CHARSET
- 186 - BALTIC_CHARSET
- 204 - RUSSIAN_CHARSET
- 222 - THAI_CHARSET
- 238 - EASTEUROPE_CHARSET
- 255 - OEM_CHARSET
Note: GDI charsets 1 and 255 are system dependent and are therefore expected to return different results across different machines. Values not on the table are treated as a reference to DEFAULT_CHARSET, equivalent to using C = 1.
For example, if you want to get the text (FROM GREEK TO UTF8), which used the ISO-8859-7 (GREEK) encoding for greek letters, in utf8, you need to encode that to utf8, interpreting the bytes as per in the GREEK code page, and then to decode that to utf8: $utfdecode($utfencode(text,161))
If you want to send the text in GREEK over IRC, mIRC will encode the bytes internally so you must encode the text in utf8, and then decode to utf8, interpreting the bytes as per in the GREEK code page: /raw -n privmsg #chan $utfdecode($utfencode(text),161)
/raw -n doesn't encode the bytes < 255.