|
|
|
Python/C API Reference Manual |
|
|
|
7.3.2.1 Built-in Codecs
Python provides a set of builtin codecs which are written in C
for speed. All of these codecs are directly usable via the
following functions.
Many of the following APIs take two arguments encoding and
errors. These parameters encoding and errors have the same semantics
as the ones of the builtin unicode() Unicode object constructor.
Setting encoding to NULL causes the default encoding to be used
which is ASCII. The file system calls should use
Py_FileSystemDefaultEncoding as the encoding for file
names. This variable should be treated as read-only: On some systems,
it will be a pointer to a static string, on others, it will change at
run-time, e.g. when the application invokes setlocale.
Error handling is set by errors which may also be set to NULL
meaning to use the default handling defined for the codec. Default
error handling for all builtin codecs is ``strict''
(ValueError is raised).
The codecs all use a similar interface. Only deviation from the
following generic ones are documented for simplicity.
These are the generic codec APIs:
PyObject* PyUnicode_Decode( | const char *s,
int size,
const char *encoding,
const char *errors) |
-
Return value:
New reference.
Create a Unicode object by decoding size bytes of the encoded
string s. encoding and errors have the same
meaning as the parameters of the same name in the
unicode() builtin function. The codec to be used is
looked up using the Python codec registry. Returns NULL if an
exception was raised by the codec.
PyObject* PyUnicode_Encode( | const Py_UNICODE *s,
int size,
const char *encoding,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size and returns
a Python string object. encoding and errors have the
same meaning as the parameters of the same name in the Unicode
encode() method. The codec to be used is looked up using
the Python codec registry. Returns NULL if an exception was
raised by the codec.
PyObject* PyUnicode_AsEncodedString( | PyObject *unicode,
const char *encoding,
const char *errors) |
-
Return value:
New reference.
Encodes a Unicode object and returns the result as Python string
object. encoding and errors have the same meaning as the
parameters of the same name in the Unicode encode() method.
The codec to be used is looked up using the Python codec registry.
Returns NULL if an exception was raised by the codec.
These are the UTF-8 codec APIs:
PyObject* PyUnicode_DecodeUTF8( | const char *s,
int size,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the UTF-8
encoded string s. Returns NULL if an exception was raised
by the codec.
PyObject* PyUnicode_EncodeUTF8( | const Py_UNICODE *s,
int size,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using UTF-8
and returns a Python string object. Returns NULL if an exception
was raised by the codec.
PyObject* PyUnicode_AsUTF8String( | PyObject *unicode) |
-
Return value:
New reference.
Encodes a Unicode objects using UTF-8 and returns the result as
Python string object. Error handling is ``strict''. Returns
NULL if an exception was raised by the codec.
These are the UTF-16 codec APIs:
PyObject* PyUnicode_DecodeUTF16( | const char *s,
int size,
const char *errors,
int *byteorder) |
-
Return value:
New reference.
Decodes length bytes from a UTF-16 encoded buffer string and
returns the corresponding Unicode object. errors (if
non-NULL) defines the error handling. It defaults to ``strict''.
If byteorder is non-NULL, the decoder starts decoding using
the given byte order:
*byteorder == -1: little endian
*byteorder == 0: native order
*byteorder == 1: big endian
and then switches according to all byte order marks (BOM) it finds
in the input data. BOMs are not copied into the resulting Unicode
string. After completion, *byteorder is set to the current
byte order at the end of input data.
If byteorder is NULL, the codec starts in native order mode.
Returns NULL if an exception was raised by the codec.
PyObject* PyUnicode_EncodeUTF16( | const Py_UNICODE *s,
int size,
const char *errors,
int byteorder) |
-
Return value:
New reference.
Returns a Python string object holding the UTF-16 encoded value of
the Unicode data in s. If byteorder is not 0
,
output is written according to the following byte order:
byteorder == -1: little endian
byteorder == 0: native byte order (writes a BOM mark)
byteorder == 1: big endian
If byteorder is 0
, the output string will always start with
the Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark
is prepended.
Note that Py_UNICODE data is being interpreted as UTF-16
reduced to UCS-2. This trick makes it possible to add full UTF-16
capabilities at a later point without comprimising the APIs.
Returns NULL if an exception was raised by the codec.
PyObject* PyUnicode_AsUTF16String( | PyObject *unicode) |
-
Return value:
New reference.
Returns a Python string using the UTF-16 encoding in native byte
order. The string always starts with a BOM mark. Error handling is
``strict''. Returns NULL if an exception was raised by the
codec.
These are the ``Unicode Escape'' codec APIs:
PyObject* PyUnicode_DecodeUnicodeEscape( | const char *s,
int size,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the
Unicode-Escape encoded string s. Returns NULL if an
exception was raised by the codec.
PyObject* PyUnicode_EncodeUnicodeEscape( | const Py_UNICODE *s,
int size,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using
Unicode-Escape and returns a Python string object. Returns NULL
if an exception was raised by the codec.
PyObject* PyUnicode_AsUnicodeEscapeString( | PyObject *unicode) |
-
Return value:
New reference.
Encodes a Unicode objects using Unicode-Escape and returns the
result as Python string object. Error handling is ``strict''.
Returns NULL if an exception was raised by the codec.
These are the ``Raw Unicode Escape'' codec APIs:
PyObject* PyUnicode_DecodeRawUnicodeEscape( | const char *s,
int size,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the
Raw-Unicode-Escape encoded string s. Returns NULL if an
exception was raised by the codec.
PyObject* PyUnicode_EncodeRawUnicodeEscape( | const Py_UNICODE *s,
int size,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using
Raw-Unicode-Escape and returns a Python string object. Returns
NULL if an exception was raised by the codec.
PyObject* PyUnicode_AsRawUnicodeEscapeString( | PyObject *unicode) |
-
Return value:
New reference.
Encodes a Unicode objects using Raw-Unicode-Escape and returns the
result as Python string object. Error handling is ``strict''.
Returns NULL if an exception was raised by the codec.
These are the Latin-1 codec APIs:
Latin-1 corresponds to the first 256 Unicode ordinals and only these
are accepted by the codecs during encoding.
PyObject* PyUnicode_DecodeLatin1( | const char *s,
int size,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the Latin-1
encoded string s. Returns NULL if an exception was raised
by the codec.
PyObject* PyUnicode_EncodeLatin1( | const Py_UNICODE *s,
int size,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using
Latin-1 and returns a Python string object. Returns NULL if an
exception was raised by the codec.
PyObject* PyUnicode_AsLatin1String( | PyObject *unicode) |
-
Return value:
New reference.
Encodes a Unicode objects using Latin-1 and returns the result as
Python string object. Error handling is ``strict''. Returns
NULL if an exception was raised by the codec.
These are the ASCII codec APIs. Only 7-bit ASCII data is
accepted. All other codes generate errors.
PyObject* PyUnicode_DecodeASCII( | const char *s,
int size,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the
ASCII encoded string s. Returns NULL if an exception
was raised by the codec.
PyObject* PyUnicode_EncodeASCII( | const Py_UNICODE *s,
int size,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using
ASCII and returns a Python string object. Returns NULL if an
exception was raised by the codec.
PyObject* PyUnicode_AsASCIIString( | PyObject *unicode) |
-
Return value:
New reference.
Encodes a Unicode objects using ASCII and returns the result as
Python string object. Error handling is ``strict''. Returns
NULL if an exception was raised by the codec.
These are the mapping codec APIs:
This codec is special in that it can be used to implement many
different codecs (and this is in fact what was done to obtain most of
the standard codecs included in the encodings package). The
codec uses mapping to encode and decode characters.
Decoding mappings must map single string characters to single Unicode
characters, integers (which are then interpreted as Unicode ordinals)
or None (meaning "undefined mapping" and causing an error).
Encoding mappings must map single Unicode characters to single string
characters, integers (which are then interpreted as Latin-1 ordinals)
or None (meaning "undefined mapping" and causing an error).
The mapping objects provided must only support the __getitem__ mapping
interface.
If a character lookup fails with a LookupError, the character is
copied as-is meaning that its ordinal value will be interpreted as
Unicode or Latin-1 ordinal resp. Because of this, mappings only need
to contain those mappings which map characters to different code
points.
PyObject* PyUnicode_DecodeCharmap( | const char *s,
int size,
PyObject *mapping,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the encoded
string s using the given mapping object. Returns
NULL if an exception was raised by the codec.
PyObject* PyUnicode_EncodeCharmap( | const Py_UNICODE *s,
int size,
PyObject *mapping,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using the
given mapping object and returns a Python string object.
Returns NULL if an exception was raised by the codec.
PyObject* PyUnicode_AsCharmapString( | PyObject *unicode,
PyObject *mapping) |
-
Return value:
New reference.
Encodes a Unicode objects using the given mapping object and
returns the result as Python string object. Error handling is
``strict''. Returns NULL if an exception was raised by the
codec.
The following codec API is special in that maps Unicode to Unicode.
PyObject* PyUnicode_TranslateCharmap( | const Py_UNICODE *s,
int size,
PyObject *table,
const char *errors) |
-
Return value:
New reference.
Translates a Py_UNICODE buffer of the given length by
applying a character mapping table to it and returns the
resulting Unicode object. Returns NULL when an exception was
raised by the codec.
The mapping table must map Unicode ordinal integers to Unicode
ordinal integers or None (causing deletion of the character).
Mapping tables need only provide the method__getitem__()
interface; dictionaries and sequences work well. Unmapped character
ordinals (ones which cause a LookupError) are left
untouched and are copied as-is.
These are the MBCS codec APIs. They are currently only available on
Windows and use the Win32 MBCS converters to implement the
conversions. Note that MBCS (or DBCS) is a class of encodings, not
just one. The target encoding is defined by the user settings on the
machine running the codec.
PyObject* PyUnicode_DecodeMBCS( | const char *s,
int size,
const char *errors) |
-
Return value:
New reference.
Creates a Unicode object by decoding size bytes of the MBCS
encoded string s. Returns NULL if an exception was
raised by the codec.
PyObject* PyUnicode_EncodeMBCS( | const Py_UNICODE *s,
int size,
const char *errors) |
-
Return value:
New reference.
Encodes the Py_UNICODE buffer of the given size using MBCS
and returns a Python string object. Returns NULL if an exception
was raised by the codec.
PyObject* PyUnicode_AsMBCSString( | PyObject *unicode) |
-
Return value:
New reference.
Encodes a Unicode objects using MBCS and returns the result as
Python string object. Error handling is ``strict''. Returns
NULL if an exception was raised by the codec.
|
|
|
Python/C API Reference Manual |
|
|
|
Release 2.3.4, documentation updated on May 20, 2004.
See About this document... for information on suggesting changes.