Introduction to i18n

Abstract

This document describes basic concepts for i18n (internationalization), how to write an internationalized software, and how to modify and internationalize a software. Handling of characters is discussed in detail. There are a few case-studies in which the author internationalized softwares such as TWM.

Copyright Notice

Copyright © 1999-2001 Tomohiro KUBOTA. Chapters and sections whose original author is not KUBOTA are copyright by their authors. Their names are written at the top of the chapter or the section.

This manual is free software; you may redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.

This is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

A copy of the GNU General Public License is available as /usr/share/common-licenses/GPL in the Debian GNU/Linux distribution or on the World Wide Web at http://www.gnu.org/copyleft/gpl.html. You can also obtain it by writing to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.

Contents

• 1 About This Document
• 1.1 Scope
• 1.2 New Versions of This Document
• 1.3 Feedback and Contributions
• 2 Introduction
• 2.1 General Concepts
• 2.2 Organization
• 3 Important Concepts for Character Coding Systems
• 3.1 Basic Terminology
• 3.2 Stateless and Stateful
• 3.3 Multibyte encodings
• 3.4 Number of Bytes, Number of Characters, and Number of Columns
• 4 Coded Character Sets And Encodings in the World
• 4.1 ASCII and ISO 646
• 4.2 ISO 8859
• 4.3 ISO 2022
• 4.3.1 EUC (Extended Unix Code)
• 4.3.2 ISO 2022-compliant Character Sets
• 4.3.3 ISO 2022-compliant Encodings
• 4.4 ISO 10646 and Unicode
• 4.4.1 UCS as a Coded Character Set
• 4.4.2 UTF as Character Encoding Schemes
• 4.4.3 Problems on Unicode
• 4.5 Other Character Sets and Encodings
• 4.5.1 Big5
• 4.5.2 UHC
• 4.5.3 Johab
• 4.5.4 HZ, aka HZ-GB-2312
• 4.5.5 GBK
• 4.5.6 GB18030
• 4.5.7 GCCS
• 4.5.8 HKSCS
• 4.5.9 Shift-JIS
• 4.5.10 VISCII
• 4.5.11 TRON
• 4.5.12 Mojikyo
• 5 Characters in Each Country
• 5.1 Japanese language / used in Japan
• 5.1.1 Characters used in Japanese
• 5.1.2 Character Sets
• 5.1.3 Encodings
• 5.1.4 How These Encodings Are Used --- Information for Programmers
• 5.1.5 Columns
• 5.1.6 Writing Direction and Combined Characters
• 5.1.7 Layout of Characters
• 5.1.8 LANG variable
• 5.1.9 Input from Keyboard
• 5.1.10 More Detailed Discussions
• 5.2 Spanish language / used in Spain, most of America and Equatorial Guinea
• 5.2.1 Characters used in Spanish
• 5.2.2 Character Sets
• 5.2.3 Codesets
• 5.2.4 How These Codesets Are Used --- Information for Programmers
• 5.2.5 Columns
• 5.2.6 Writing Direction
• 5.2.7 Layout of Characters
• 5.2.8 LANG variable
• 5.2.9 Input from Keyboard
• 5.2.10 More Detailed Discussions
• 5.3 Languages with Cyrillic script
• 6 LOCALE technology
• 6.1 Locale Categories and setlocale()
• 6.2 Locale Names
• 6.3 Multibyte Characters and Wide Characters
• 6.4 Unicode and LOCALE technology
• 6.5 nl_langinfo() and iconv()
• 6.6 Limit of Locale technology
• 7 Output to Display
• 7.1 Console Softwares
• 7.1.1 Encoding
• 7.1.2 Number of Columns
• 7.2 X Clients
• 7.2.1 Xlib programming
• 7.2.2 Athena widgets
• 7.2.3 Gtk and Gnome
• 7.2.4 Qt and KDE
• 8 Input from Keyboard
• 8.1 Non-X Softwares
• 8.2 X Softwares
• 8.2.1 Developing XIM clients
• 8.2.2 Examples of XIM softwares
• 8.2.3 Using XIM softwares
• 8.3 Emacsen
• 9 Internal Processing and File I/O
• 9.1 Stream I/O of Characters
• 9.2 Character Classification
• 9.3 Length of String
• 9.4 Extraction of Characters
• 10 the Internet
• 10.1 Mail/News
• 10.2 WWW
• 11 Libraries and Components
• 11.1 Gettext and Translation
• 11.1.1 Gettext-ization of A Software
• 11.1.2 Translation
• 11.2 Readline Library
• 11.3 Ncurses Library
• 12 Softwares Written in Other than C/C++
• 12.1 Fortran
• 12.2 Pascal
• 12.3 Perl
• 12.4 Python
• 12.5 Ruby
• 12.6 Tcl/Tk
• 12.7 Java
• 12.8 Shell Script
• 12.9 Lisp
• 13 Examples of I18N
• 13.1 TWM -- usage of XFontSet instead of XFontStruct
• 13.1.1 Introduction
• 13.1.2 Locale Setting - A Routine Work
• 13.1.3 Font Preparation
• 13.1.4 Automatic Font Guessing
• 13.1.5 Font Preparation (continued)
• 13.1.6 Drawing Text using MyFont
• 13.1.7 Geting Size of Texts
• 13.1.8 Getting Window Titles
• 13.1.9 Getting Icon Names
• 13.1.10 Configuration File Parser
• 13.2 8bit-clean-ize of Minicom
• 13.2.1 8bit-clean-ize
• 13.2.2 Not to break continuity of multibyte characters
• 13.2.3 Catalog in EUC-JP and SHIFT-JIS
• 13.3 user-ja -- two sets of messages in ASCII and native codeset in the same language
• 13.3.1 Introduction
• 13.3.2 Strategy
• 13.3.3 Implementation
• 13.4 A Quasi-Wrapper to Internationalize Text Output of X Clients
• 13.4.1 Introduction
• 13.4.2 Strategy
• 13.4.3 Usage of the wrapper
• 13.4.4 The Header File of the Wrapper
• 13.4.5 The Source File of the Wrapper
• 14 References

Introduction to i18n