Symbian
Symbian OS Library

SYMBIAN OS V9.3

[Index] [Spacer] [Previous] [Next]



Location: charconv.h
Link against: charconv.lib

Class CCnvCharacterSetConverter

class CCnvCharacterSetConverter : public CBase;

Description

Converts text between Unicode and other character sets.

The first stage of the conversion is to specify the non-Unicode character set being converted to or from. This is done by calling one of the overloads of PrepareToConvertToOrFromL().

The second stage is to convert the text, using one of the overloads of ConvertFromUnicode() or ConvertToUnicode().

Where possible the first documented overload of PrepareToConvertToOrFromL() should be used because the second overload panics if the specified character set is not available: the first overload simply returns whether the character set is available or not available. However if the conversions are to be performed often, or if the user must select the character set for the conversion from a list, the second overload may be more appropriate.

The first overload is less efficient than the second, because it searches through the file system for the selected character set every time it is invoked. The second overload searches through an array of all available character sets. In this method, the file system need only be searched once - when CreateArrayOfCharacterSetsAvailableLC() or CreateArrayOfCharacterSetsAvailableL() is used to create the array.

The conversion functions allow users of this class to perform partial conversions on an input descriptor, handling the situation where the input descriptor is truncated mid way through a multi-byte character. This means that you do not have to guess how big to make the output descriptor for a given input descriptor, you can simply do the conversion in a loop using a small output descriptor. The ability to handle truncated descriptors also allows users of the class to convert information received in chunks from an external source.

The class also provides a number of utility functions.

Derivation

Members

Defined in CCnvCharacterSetConverter:
AsciiConversionData(), AutoDetectCharSetL(), AutoDetectCharacterSetL(), ConvertCharacterSetIdentifierToMibEnumL(), ConvertCharacterSetIdentifierToStandardNameL(), ConvertFromUnicode(), ConvertFromUnicode(), ConvertFromUnicode(), ConvertFromUnicode(), ConvertMibEnumOfCharacterSetToIdentifierL(), ConvertStandardNameOfCharacterSetToIdentifierL(), ConvertToUnicode(), ConvertToUnicode(), ConvertToUnicode(), ConvertibleToCharSetL(), ConvertibleToCharacterSetL(), CreateArrayOfCharacterSetsAvailableL(), CreateArrayOfCharacterSetsAvailableLC(), DoConvertFromUnicode(), DoConvertFromUnicode(), DoConvertToUnicode(), DoConvertToUnicode(), EAvailable, EBigEndian, EDowngradeExoticLineTerminatingCharactersToCarriageReturnLineFeed, EDowngradeExoticLineTerminatingCharactersToJustLineFeed, EErrorIllFormedInput, EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable, EInputConversionFlagAppend, EInputConversionFlagAssumeStartInDefaultCharacterSet, EInputConversionFlagMustEndInDefaultCharacterSet, EInputConversionFlagStopAtFirstUnconvertibleCharacter, ELittleEndian, ELowestThreshold, ENotAvailable, EOutputConversionFlagInputIsTruncated, GetDowngradeForExoticLineTerminatingCharacters(), KStateDefault, NewL(), NewLC(), PrepareToConvertToOrFromL(), PrepareToConvertToOrFromL(), SCharacterSet, SetDefaultEndiannessOfForeignCharacters(), SetDowngradeForExoticLineTerminatingCharacters(), SetMaxCacheSize(), SetReplacementForUnconvertibleUnicodeCharactersL(), TArrayOfAscendingIndices, TAvailability, TDowngradeForExoticLineTerminatingCharacters, TEndianness, TError, anonymous, anonymous, anonymous, anonymous, ~CCnvCharacterSetConverter()

Inherited from CBase:
Delete(), Extension_(), operator new()


Construction and destruction


NewL()

static IMPORT_C CCnvCharacterSetConverter *NewL();

Description

Allocates and constructs a CCnvCharacterSetConverter object. If there is insufficient memory to create the object, the function leaves.

Since the memory is allocated on the heap, objects of this type should be destroyed using the delete operator when the required conversions are complete.

Return value

CCnvCharacterSetConverter *

The newly created object.


NewLC()

static IMPORT_C CCnvCharacterSetConverter *NewLC();

Description

Allocates and constructs a CCnvCharacterSetConverter object, and leaves the object on the cleanup stack. If there is insufficient memory to create the object, the function leaves.

Since the memory is allocated on the heap, objects of this type should be destroyed using either the CleanupStack::Pop() function and then the delete operator, or the CleanupStack::PopAndDestroy() function.

Return value

CCnvCharacterSetConverter *

The newly created object.


~CCnvCharacterSetConverter()

virtual IMPORT_C ~CCnvCharacterSetConverter();

Description

The destructor frees all resources owned by the object, prior to its destruction.

[Top]


Member functions


CreateArrayOfCharacterSetsAvailableL()

static IMPORT_C CArrayFix< SCharacterSet > *CreateArrayOfCharacterSetsAvailableL(RFs &aFileServerSession);

Description

Creates an array identifying all the character sets for which conversion is available. These can be character sets for which conversion is built into Symbian OS, or they may be character sets for which conversion is implemented by a plug-in DLL.

The array returned can be used by one of the PrepareToConvertToOrFromL() overloads to provide a list of all the character sets available for conversion. The caller of this function is responsible for deleting the array, and should not modify it.

Not all encoders returned will be suitable for conversion from Unicode. Such encoders have no name and no MIB enum and so will generally not be understood by a receiving process. The function ConvertCharacterSetIdentifierToMibEnumL can be used to determine whether this is the case or not.

Parameters

RFs &aFileServerSession

A file server session. This is no longer used, but is kept for Binary Compatibility.

Return value

CArrayFix< SCharacterSet > *

An array identifying all supported character sets.


CreateArrayOfCharacterSetsAvailableLC()

static IMPORT_C CArrayFix< SCharacterSet > *CreateArrayOfCharacterSetsAvailableLC(RFs &aFileServerSession);

Description

Creates an array identifying all the character sets for which conversion is available and pushes a pointer to it onto the cleanup stack. These can be character sets for which conversion is built into Symbian OS, or they may be character sets for which conversion is implemented by a plug-in DLL.

The array returned can be used by one of the PrepareToConvertToOrFromL() overloads to provide a list of all the character sets available for conversion. The caller of this function is responsible for deleting the array, and should not modify it.

Not all encoders returned will be suitable for conversion from Unicode. Such encoders have no name and no MIB enum and so will generally not be understood by a receiving process. The function ConvertCharacterSetIdentifierToMibEnumL can be used to determine whether this is the case or not.

This is a static function which uses ECOM functionality. It cleans up ECOM by calling FinalClose()

Parameters

RFs &aFileServerSession

A file server session. This is no longer required, kept for Binary Compatibilty.

Return value

CArrayFix< SCharacterSet > *

An array of references to all supported character sets.


ConvertStandardNameOfCharacterSetToIdentifierL()

IMPORT_C TUint ConvertStandardNameOfCharacterSetToIdentifierL(const TDesC8 &aStandardNameOfCharacterSet, RFs &aFileServerSession);

Description

Gets the UID of a character set identified by its Internet-standard name (the matching is case-insensitive).

If the character set specified is not one for which Symbian OS provides built-in conversion, the function searches the file system for plug-ins which implement the conversion and which provide the name-to-UID mapping information.

Parameters

const TDesC8 &aStandardNameOfCharacterSet

Internet-standard name of a character set.

RFs &aFileServerSession

Connection to a file server session.

Return value

TUint

The UID for the character set if the character set with a Internet-standard name has a known name and MIB enum.Otherwise zero is returned.


ConvertCharacterSetIdentifierToStandardNameL()

IMPORT_C HBufC8 *ConvertCharacterSetIdentifierToStandardNameL(TUint aCharacterSetIdentifier, RFs &aFileServerSession);

Description

Returns the Internet-standard name of a character set identified in Symbian OS by a UID.

If the character set specified is not one for which Symbian OS provides built-in conversion, the file system is searched for plug-ins which implement the conversion, hence the need for a file server session.

Parameters

TUint aCharacterSetIdentifier

The UID of the character set.

RFs &aFileServerSession

A file server session.

Return value

HBufC8 *

The Internet-standard name for the character set if the character set with an identifier has a known name and MIB enum.Otherwise NULL is returned.


ConvertMibEnumOfCharacterSetToIdentifierL()

IMPORT_C TUint ConvertMibEnumOfCharacterSetToIdentifierL(TInt aMibEnumOfCharacterSet, RFs &aFileServerSession);

Description

Converts a MIB enum value to the UID value of the character set.

If the character set identified is not one for which Symbian OS provides built-in conversion, the function searches the file system for plug-ins which implement the conversion and which provide the MIB enum-to-UID mapping information.

Parameters

TInt aMibEnumOfCharacterSet

The MIB enum value of the character set.

RFs &aFileServerSession

Connection to a file server session.

Return value

TUint

TThe UID of the character set if the character set with a Mib enum has a known name and MIB enum.Otherwise zero is returned.


ConvertCharacterSetIdentifierToMibEnumL()

IMPORT_C TInt ConvertCharacterSetIdentifierToMibEnumL(TUint aCharacterSetIdentifier, RFs &aFileServerSession);

Description

Converts the UID of a character set to its MIB enum value.

If the character set identified is not one for which Symbian OS provides built-in conversion, the function searches the file system for plug-ins which implement the conversion and which provide the UID-to-MIB enum mapping information.

Parameters

TUint aCharacterSetIdentifier

The UID of the character set.

RFs &aFileServerSession

Connection to a file server session.

Return value

TInt

The MIB enum value of the character set if the character set with Identifier has a known name and MIB enum.Otherwise zero is returned.


PrepareToConvertToOrFromL()

IMPORT_C void PrepareToConvertToOrFromL(TUint aCharacterSetIdentifier, const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable, RFs &aFileServerSession);

Description

Specifies the character set to convert to or from. aCharacterSetIdentifier is a UID which identifies a character set. It can be one of the character sets for which conversion is built into Symbian OS, or it may be a character set for which the conversion is implemented by a plug-in DLL.

The function searches the character set array specified (aArrayOfCharacterSetsAvailable). This is an array containing all of the character sets for which conversion is available. It is created by calling CreateArrayOfCharacterSetsAvailableL() or CreateArrayOfCharacterSetsAvailableLC(). You should be sure that conversion is available for aCharacterSetIdentifier, because if not, a panic occurs. Otherwise, use the other overload of this function.

Either this function or its overload, must be called before using the conversion functions ConvertFromUnicode() or ConvertToUnicode().

Unlike the other overload, this function does not search the file system for plug-in conversion DLLs, (unless aArrayOfCharacterSetsAvailable is NULL). This function should be used if conversions are to be performed often, or if the conversion character set is to be selected by the user. Generating the array of all the available character sets once and searching though it is more efficient than the method used by the other overload, in which the file system may be searched every time it is invoked.

Notes:

The file server session argument is used to open the required character set conversion data file.

The array passed to this function can also be used to provide a list from which a user can select the desired conversion character set.

Parameters

TUint aCharacterSetIdentifier

The UID of the non-Unicode character set from or to which to convert. Must not be zero, or a panic occurs.

const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable

Array of all character sets for which conversion is available, created by either CreateArrayOfCharacterSetsAvailableLC() or CreateArrayOfCharacterSetsAvailableL().

RFs &aFileServerSession

A file server session. No longer used, kept for Binary Compatibility


PrepareToConvertToOrFromL()

IMPORT_C TAvailability PrepareToConvertToOrFromL(TUint aCharacterSetIdentifier, RFs &aFileServerSession);

Description

Specifies the character set to convert to or from. aCharacterSetIdentifier is a UID which identifies a character set. It can be one of the character sets for which conversion is built into Symbian OS, or it may be a character set for which conversion is implemented by a plug-in DLL. In the latter case, the function searches through the file system for the DLL which implements the character conversion.

Either this function or its overload must be called before using the conversion functions ConvertFromUnicode() or ConvertToUnicode().

This overload of the function is simpler to use than the other and does not panic if the character set with the specified UID is not available at run timeÂ, it simply returns ENotAvailable. It should be used when the conversion character set is specified within the text object being converted, e.g. an email message, or an HTML document. If the character set is not specified, the user must be presented with a list of all available sets, so it makes sense to use the other overload.

The function may need to search the file system each time it is called. If conversion takes place repeatedly over a short period, it may be more efficient to use the other overload.

Notes:

Although the other overload of this function is more efficient, if the character set is one for which conversion is built into Symbian OS, the difference in speed is negligible.

Parameters

TUint aCharacterSetIdentifier

The UID of the non-Unicode character set from or to which to convert. Must not be zero, or a panic occurs.

RFs &aFileServerSession

A file server session. No longer used, kept for Binary Compatibility

Return value

TAvailability

The availability of the specified character set. If EAvailable is returned, then the conversion functions ConvertToUnicode() and ConvertFromUnicode() will use aCharacterSetIdentifier as the foreign character set. If ENotAvailable is returned, then the foreign character set will either be undefined (and trying to use the conversion functions will cause a panic), or if it has previously been set, it will remain unchanged.


SetDefaultEndiannessOfForeignCharacters()

IMPORT_C void SetDefaultEndiannessOfForeignCharacters(TEndianness aEndianness);

Description

Sets the default endian-ness used by the ConvertFromUnicode() and ConvertToUnicode() functions to convert between Unicode and non-Unicode character sets.

The endian-ness of a multi-byte character set may be defined in the character set definition or, as in the case of UCS-2, be operating system dependent. If the endian-ness of the current character set is defined by the character set itself, then the default endian-ness specified by this function is ignored.

Notes:

The issue of endian-ness does not apply to single byte character sets as there is no byte order.

This function should be called (if at all) after calling PrepareToConvertToOrFromL() and before calling ConvertFromUnicode() and/or ConvertToUnicode().

Parameters

TEndianness aEndianness

The default endian-ness of the current character set.


SetDowngradeForExoticLineTerminatingCharacters()

IMPORT_C void SetDowngradeForExoticLineTerminatingCharacters(TDowngradeForExoticLineTerminatingCharacters aDowngradeForExoticLineTerminatingCharacters);

Description

Sets whether the Unicode 'line separator' and 'paragraph separator' characters (0x2028 and 0x2029 respectively) should be converted into a carriage return / line feed pair, or into a line feed only when converting from Unicode into a foreign character set. This applies to all foreign character sets that do not contain a direct equivalent of these Unicode character codes.

By default, line and paragraph separators are converted into a CR/LF pair. This function should be called (if at all) after calling PrepareToConvertToOrFromL() and before calling ConvertFromUnicode() and/or ConvertToUnicode().

Parameters

TDowngradeForExoticLineTerminatingCharacters aDowngradeForExoticLineTerminatingCharacters

Specify EDowngradeExoticLineTerminatingCharactersToCarriageReturnLineFeed if line/paragraph separators should be converted into a carriage return and line feed combination and EDowngradeExoticLineTerminatingCharactersToJustLineFeed if they should be converted into line feeds only. Any other value causes the function to panic.


SetReplacementForUnconvertibleUnicodeCharactersL()

IMPORT_C void SetReplacementForUnconvertibleUnicodeCharactersL(const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters);

Description

Sets the character used to replace unconvertible characters in the output descriptor, when converting from Unicode into another character set.

The default replacement for unconvertible Unicode characters is specified in the conversion data for the character set. The replacement text which is set using this function overrides the default value.

Notes:

If the replacement character is multi-byte, and its endian-ness is undefined in the character set, then its byte order is taken by default to be little-endian.

PrepareToConvertToOrFromL() undoes the effect of any previous calls to this function. So, to have any effect, this function should be called between the PrepareToConvertToOrFromL() call and the subsequent ConvertFromUnicode() call or calls.

The value only applies when converting from Unicode to another character set. In Unicode, the code for 'unknown character'is always 0xFFFD.

Parameters

const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters

The single character which is to be used to replace unconvertible characters.


ConvertFromUnicode()

IMPORT_C TInt ConvertFromUnicode(TDes8 &aForeign, const TDesC16 &aUnicode) const;

Description

Converts text encoded in the Unicode character set (UCS-2) into other character sets.

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of characters that could not be converted. The third overload converts the text, gets the number of characters that could not be converted, and also gets the index of the first character that could not be converted. A fourth overload was introduced in v6.0 see below.All overloads cause a panic if no target character set has been selected to convert to (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the output descriptor.Notes:A sixteen-bit descriptor is used to hold the source Unicode encoded text, and an eight-bit descriptor is used to hold the converted non-Unicode text. Eight-bit descriptors are used because non-Unicode character sets may use a single byte per character (e.g. Code Page 1252) or more than one byte per character (e.g. GB 2312-80) or even a variable number of bytes per character (e.g. Shift-JIS).The function will fail to convert all the input descriptor if the output descriptor is not long enough to hold all the text.Unicode characters cannot be converted if there is no equivalent for them in the target character set. This does not stop the conversion, the missing character is simply replaced by the character in the target character set which represents unknown characters. This default unknown character can be changed using SetReplacementForUnconvertibleUnicodeCharactersL().

Parameters

TDes8 &aForeign

On return, contains the converted text in a non-Unicode character set.

const TDesC16 &aUnicode

The source Unicode text to be converted.

Return value

TInt

The number of unconverted characters left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.


ConvertFromUnicode()

IMPORT_C TInt ConvertFromUnicode(TDes8 &aForeign, const TDesC16 &aUnicode, TInt &aNumberOfUnconvertibleCharacters) const;

Description

Parameters

TDes8 &aForeign

const TDesC16 &aUnicode

TInt &aNumberOfUnconvertibleCharacters

Return value

TInt


ConvertFromUnicode()

IMPORT_C TInt ConvertFromUnicode(TDes8 &aForeign, const TDesC16 &aUnicode, TInt &aNumberOfUnconvertibleCharacters, TInt &aIndexOfFirstUnconvertibleCharacter) const;

Description

Converts text encoded in the Unicode character set (UCS-2) into other character sets.

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of characters that could not be converted. The third overload converts the text, gets the number of characters that could not be converted, and also gets the index of the first character that could not be converted. A fourth overload was introduced in v6, see below.

All overloads cause a panic if no target character set has been selected to convert to (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the output descriptor.

Notes:

A sixteen-bit descriptor is used to hold the source Unicode encoded text, and an eight-bit descriptor is used to hold the converted non-Unicode text. Eight-bit descriptors are used because non-Unicode character sets may use a single byte per character (e.g. Code Page 1252) or more than one byte per character (e.g. GB 2312-80) or even a variable number of bytes per character (e.g. Shift-JIS).

The function will fail to convert all the input descriptor if the output descriptor is not long enough to hold all the text.

Unicode characters cannot be converted if there is no equivalent for them in the target character set. This does not stop the conversion, the missing character is simply replaced by the character in the target character set which represents unknown characters. This default unknown character can be changed using SetReplacementForUnconvertibleUnicodeCharactersL().

Parameters

TDes8 &aForeign

On return, contains the converted text in a non-Unicode character set.

const TDesC16 &aUnicode

The source Unicode text to be converted.

TInt &aNumberOfUnconvertibleCharacters

On return contains the number of characters which could not be converted.

TInt &aIndexOfFirstUnconvertibleCharacter

On return, contains the index of the first character in the input text that could not be converted. The value is negative if all characters were converted.

Return value

TInt

The number of unconverted characters left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.


ConvertFromUnicode()

IMPORT_C TInt ConvertFromUnicode(TDes8 &aForeign, const TDesC16 &aUnicode, TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters) const;

Description

Converts Unicode text into another character set.

Differs from the other overloads of this function by returning the indices of all of the characters in the source Unicode text which could not be converted.

Parameters

TDes8 &aForeign

On return, contains the converted text in a non-Unicode character set.

const TDesC16 &aUnicode

The source Unicode text to be converted.

TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters

On return, holds the indices of each Unicode character in the source text which could not be converted.

Return value

TInt

The number of unconverted characters left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.


ConvertToUnicode()

IMPORT_C TInt ConvertToUnicode(TDes16 &aUnicode, const TDesC8 &aForeign, TInt &aState) const;

Description

Converts text encoded in a non-Unicode character set into the Unicode character set (UCS-2).

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of bytes in the input string that could not be converted. The third overload converts the text, gets the number of bytes that could not be converted, and also gets the index of the first byte that could not be converted.All overloads cause a panic if no source character set has been selected to convert from (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the input descriptor.Notes: Since Unicode is intended to be the superset of all character sets, the function should usually report zero unconverted characters. Unconvertible characters will exist if the input descriptor contains illegal characters, i.e. values not in the selected non-Unicode character set.The presence of illegal characters does not stop the conversion. The missing character is simply replaced by the Unicode character which represents unknown characters (0xFFFD).If the source text consists solely of a character that is not complete, the function returns EErrorIllFormedInput. The reason for this is to prevent the possibility of the calling code getting into a infinite loop.

Parameters

TDes16 &aUnicode

On return, contains the converted text in the Unicode character aForeign The non-Unicode source text to be converted.

const TDesC8 &aForeign

TInt &aState

This is used to save state information across multiple calls to ConvertToUnicode(). You should initialise the value to KStateDefault, and then do not change it in a series of related calls.

Return value

TInt

The number of unconverted bytes left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.


ConvertToUnicode()

IMPORT_C TInt ConvertToUnicode(TDes16 &aUnicode, const TDesC8 &aForeign, TInt &aState, TInt &aNumberOfUnconvertibleCharacters) const;

Description

Parameters

TDes16 &aUnicode

const TDesC8 &aForeign

TInt &aState

TInt &aNumberOfUnconvertibleCharacters

Return value

TInt


ConvertToUnicode()

IMPORT_C TInt ConvertToUnicode(TDes16 &aUnicode, const TDesC8 &aForeign, TInt &aState, TInt &aNumberOfUnconvertibleCharacters, TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter) const;

Description

Converts text encoded in a non-Unicode character set into the Unicode character set (UCS-2).

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of bytes in the input string that could not be converted. The third overload converts the text, gets the number of bytes that could not be converted, and also gets the index of the first byte that could not be converted.

All overloads cause a panic if no source character set has been selected to convert from (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the input descriptor.

Notes:

Since Unicode is intended to be the superset of all character sets, the function should usually report zero unconverted characters. Unconvertible characters will exist if the input descriptor contains illegal characters, i.e. values not in the selected non-Unicode character set.

The presence of illegal characters does not stop the conversion. The missing character is simply replaced by the Unicode character which represents unknown characters (0xFFFD).

If the source text consists solely of a character that is not complete, the function returns EErrorIllFormedInput. The reason for this is to prevent the possibility of the calling code getting into a infinite loop.

Parameters

TDes16 &aUnicode

On return, contains the converted text in the Unicode character set.

const TDesC8 &aForeign

The non-Unicode source text to be converted.

TInt &aState

This is used to save state information across multiple calls to ConvertToUnicode(). You should initialise the value to KStateDefault, and then do not change it in a series of related calls.

TInt &aNumberOfUnconvertibleCharacters

On return, contains the number of bytes which were not converted.

TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter

On return, the index of the first byte of the first unconvertible character. For instance if the first character in the input descriptor (aForeign) could not be converted, then this parameter is set to the first byte of that character, i.e. zero. A negative value is returned if all the characters were converted.

Return value

TInt

The number of unconverted bytes left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.


AutoDetectCharacterSetL()

static IMPORT_C void AutoDetectCharacterSetL(TInt &aConfidenceLevel, TUint &aCharacterSetIdentifier, const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable, const TDesC8 &aSample);

Pre-Condition

Description

Deprecated

Post-Condition

Parameters

TInt &aConfidenceLevel

TUint &aCharacterSetIdentifier

const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable

const TDesC8 &aSample


AutoDetectCharSetL()

IMPORT_C void AutoDetectCharSetL(TInt &aConfidenceLevel, TUint &aCharacterSetIdentifier, const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable, const TDesC8 &aSample);

Description

Attempts to determine the character set of the sample text from those supported on the phone.

For each of the available character sets, its implementation of IsInThisCharacterSetL() is called. The character set which returns the highest confidence level (i.e. which generates the fewest 0xFFFD Unicode replacement characters) is returned in aCharacterSetIdentifier.

This function merely determines if the sample text is convertible with this converter: it does no textual analysis on the result. Therefore, this function is not capable of differentiating between very similar encodings (for example the different ISO 8859 variants).

Any code making use of this function should provide a way for the user to override the selection that this function makes.

Please note that the operation of this function is slow.It takes no account of the usual ontext that would be used in guessing a character set (for example, the language that is expected to be encoded or the transport used). For situations where such context is known, a faster, more accurate solution is advisable.

To improve a performance of autodetection, a size (default is one) of interface proxy cache should be increased (see SetCharacterSetCacheSize()).However a boost of performance will not be visible within a first funtion call because during this first call character sets are loaded to a cache. Once created it will be preserved until CCnvCharacterSetConverter object is destroyed.

This is a static function which uses ECOM functionality. It cleans up ECOM by calling FinalClose()

Parameters

TInt &aConfidenceLevel

Set by the function to a value between 0 and 100. 0 indicates the function has no idea what character set aSample is encoded in. In this case, aCharacterSetIdentifier is undefined. 100 indicates total confidence that aCharacterSetIdentifier is the character set of aSample.

TUint &aCharacterSetIdentifier

On return, the UID of the best available character set for the sample text aSample. Character set UIDs are defined in charconv.h.

const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable

The array of character sets available on the phone. If this is not already available, it can be created using CreateArrayOfCharacterSetsAvailableL() or CreateArrayOfCharacterSetsAvailableLC().

const TDesC8 &aSample

The non-Unicode sample text string.


ConvertibleToCharacterSetL()

static IMPORT_C void ConvertibleToCharacterSetL(TInt &aConfidenceLevel, const TUint aCharacterSetIdentifier, const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable, const TDesC8 &aSample);

Pre-Condition

Description

Deprecated

Post-Condition

Parameters

TInt &aConfidenceLevel

const TUint aCharacterSetIdentifier

const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable

const TDesC8 &aSample


ConvertibleToCharSetL()

IMPORT_C void ConvertibleToCharSetL(TInt &aConfidenceLevel, const TUint aCharacterSetIdentifier, const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable, const TDesC8 &aSample);

Description

Given a character set UID aCharacterSetIdentifier, ConvertibleToCharacterSetL returns the likelihood that aSample is encoded in that character set. It goes through the array of character sets aArrayOfCharacterSetsAvailable and searches for the character set matching aCharacterSetIdentifier. The character sets IsInThisCharacterSetL function is called to determine the probability of it being encoded in that character set.

This is a static function which uses ECOM functionality. It cleans up ECOM by calling FinalClose()

Parameters

TInt &aConfidenceLevel

Set by the function to a value between 0 and 100. It indicates the likelihood that aSample is encoded in aCharacterSetIdentifier.

const TUint aCharacterSetIdentifier

the likelihood of aSample being encoded in that character set.

const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable

The array of character sets available on the device. If this is not already available, it can be created using CreateArrayOfCharacterSetsAvailableL() or CreateArrayOfCharacterSetsAvailableLC().

const TDesC8 &aSample

The non-Unicode sample text string.


SetMaxCacheSize()

IMPORT_C void SetMaxCacheSize(TInt aSize);

Description

The method sets the max size of the internal character set converter cache. The cache is used mainly to improve the performance of AutoDetectCharSetL() calls. It caches loaded converter implementations. The next time when a specific implementation is needed, a search will be done in the cache if this implementation is already loaded and if it is there, the cached implementation will be used. SetMaxCacheSize() call is used to limit the max cache size, because the loaded implementatiions may consume a lot of the system resources (memory for example). By default (if SetMaxCacheSize() is never called) the max cache size is limited to 32 entries. Note: Setting very small cache size will impact the overall performance of CHARCONV functions. If the choosen cache size is less than the number of existing character set converter implementations, there will be no performance gain or it will be far beyond client's expectations. For best performance the choosen cache size should be bigger or equal to the number of the existing character set converter implementations.

Parameters

TInt aSize

The new max cache size. It must be bigger or equal to 4.

Panic codes

User::Invariant()

if the new max cache size is less than 4.


DoConvertFromUnicode()

static IMPORT_C TInt DoConvertFromUnicode(const SCnvConversionData &aConversionData, TEndianness aDefaultEndiannessOfForeignCharacters, const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters, TDes8 &aForeign, const TDesC16 &aUnicode, TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters);

Description

Converts Unicode text into another character set. The Unicode text specified in aUnicode is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aForeign.

Note

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertFromUnicode(). Users of the Character Conversion API should use one of the overloads of ConvertFromUnicode() instead.

Parameters

const SCnvConversionData &aConversionData

The conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).

TEndianness aDefaultEndiannessOfForeignCharacters

The default endian-ness to use when writing the characters in the foreign character set. If an endian-ness for foreign characters is specified in aConversionData (i.e. not SCnvConversionData::EUnspecified), then that value is used and the value of aDefaultEndiannessOfForeignCharacters is ignored.

const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters

The single character which is to be used to replace unconvertible characters.

TDes8 &aForeign

On return, contains the converted text in a non-Unicode character set.

const TDesC16 &aUnicode

The source Unicode text to be converted.

TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters

On return holds the indices of each Unicode character in the source text which could not be converted (because the target character set does not have an equivalent character).

Return value

TInt

The number of unconverted characters left at the end of the input descriptor (e.g. because aForeign was not long enough to hold all the text), or a negative error value, as defined in TError.


DoConvertFromUnicode()

static IMPORT_C TInt DoConvertFromUnicode(const SCnvConversionData &aConversionData, TEndianness aDefaultEndiannessOfForeignCharacters, const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters, TDes8 &aForeign, const TDesC16 &aUnicode, TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters, TUint &aOutputConversionFlags, TUint aInputConversionFlags);

Description

Converts Unicode text into another character set. The Unicode text specified in aUnicode is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aForeign.

This overload differs from the previous one in that it allows the caller to specify flags which give more control over the conversion.

Note

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertFromUnicode(). Users of the Character Conversion API should use one of the overloads of ConvertFromUnicode() instead.

Parameters

const SCnvConversionData &aConversionData

The conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).

TEndianness aDefaultEndiannessOfForeignCharacters

The default endian-ness to use when writing the characters in the foreign character set. If an endian-ness for foreign characters is specified in aConversionData (i.e. not SCnvConversionData::EUnspecified), then that value is used and the value of aDefaultEndiannessOfForeignCharacters is ignored.

const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters

The single character which is to be used to replace unconvertible characters. If aInputConversionFlags is set to EInputConversionFlagStopAtFirstUnconvertibleCharacter, this replacement character is used to replace the first unconvertible character, then the conversion will stop.

TDes8 &aForeign

On return, contains the converted text in a non-Unicode character set. This may already contain some text. If it does, and if aInputConversionFlags specifies EInputConversionFlagAppend, then the converted text is appended to this descriptor.

const TDesC16 &aUnicode

The source Unicode text to be converted.

TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters

On return holds the indices of each Unicode character in the source descriptor aUnicode which could not be converted (because the target character set does not have an equivalent character).

TUint &aOutputConversionFlags

If the input descriptor ended in a truncated sequence, e.g. the first half of a Unicode surrogate pair, aOutputConversionFlags returns with the EOutputConversionFlagInputIsTruncated flag set.

TUint aInputConversionFlags

Specify EInputConversionFlagAppend to append the text in aUnicode to aForeign. Specify EInputConversionFlagStopAtFirstUnconvertibleCharacter to stop converting when the first unconvertible character is reached. Specify EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable to prevent the function from returning the error-code EErrorIllFormedInput when the input descriptor consists of nothing but a truncated sequence.

Return value

TInt

The number of unconverted characters left at the end of the input descriptor (e.g. because aForeign was not long enough to hold all the text), or a negative error value, as defined in TError.


DoConvertToUnicode()

static IMPORT_C TInt DoConvertToUnicode(const SCnvConversionData &aConversionData, TEndianness aDefaultEndiannessOfForeignCharacters, TDes16 &aUnicode, const TDesC8 &aForeign, TInt &aNumberOfUnconvertibleCharacters, TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter);

Description

Converts non-Unicode text into Unicode. The non-Unicode text specified in aForeign is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aUnicode.

Notes:

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertToUnicode(). Ordinary users of the Character Conversion API should use one of the overloads of ConvertToUnicode() instead.

The last two arguments return information about unconverted characters. Because Unicode is intended to cover all possible characters, these rarely report anything other than zero characters. However they report the existence of unconvertible characters if the input descriptor aForeign contains illegal characters, i.e. values not in the foreign character set.

Parameters

const SCnvConversionData &aConversionData

The conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).

TEndianness aDefaultEndiannessOfForeignCharacters

The default endian-ness of the foreign characters. If an endian-ness for foreign characters is specified in aConversionData, then that is used instead and the value of aDefaultEndiannessOfForeignCharacters is ignored.

TDes16 &aUnicode

On return, contains the text converted into Unicode.

const TDesC8 &aForeign

The non-Unicode source text to be converted.

TInt &aNumberOfUnconvertibleCharacters

On return, contains the number of characters in aForeign which were not converted. Characters which cannot be converted are output as Unicode replacement characters (0xFFFD).

TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter

On return, the index of the first byte of the first unconvertible character. For instance if the first character in the input descriptor (aForeign) could not be converted, then this parameter is set to the first byte of that character, i.e. zero. A negative value is returned if all the characters were converted.

Return value

TInt

The number of unconverted bytes left at the end of the input descriptor, or a negative error value, as defined in TError.


DoConvertToUnicode()

static IMPORT_C TInt DoConvertToUnicode(const SCnvConversionData &aConversionData, TEndianness aDefaultEndiannessOfForeignCharacters, TDes16 &aUnicode, const TDesC8 &aForeign, TInt &aNumberOfUnconvertibleCharacters, TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter, TUint &aOutputConversionFlags, TUint aInputConversionFlags);

Description

Converts non-Unicode text into Unicode. The non-Unicode text specified in aForeign is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aUnicode.

This overload differs from the previous one in that it allows the caller to specify flags which give more control over the conversion.

Notes:

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertToUnicode(). Ordinary users of the Character Conversion API should use one of the overloads of ConvertToUnicode() instead.

The aNumberOfUnconvertibleCharacters and aIndexOfFirstByteOfFirstUnconvertibleCharacter arguments return information about unconverted characters. Because Unicode is intended to cover all possible characters, these rarely report anything other than zero characters. However they report the existence of unconvertible characters if the input descriptor aForeign contains illegal characters, i.e. values not in the foreign character set.

Parameters

const SCnvConversionData &aConversionData

The conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).

TEndianness aDefaultEndiannessOfForeignCharacters

The default endian-ness of the foreign characters. If an endian-ness for foreign characters is specified in aConversionData, then that is used instead and the value of aDefaultEndiannessOfForeignCharacters is ignored.

TDes16 &aUnicode

On return, contains the text converted into Unicode.

const TDesC8 &aForeign

The non-Unicode source text to be converted.

TInt &aNumberOfUnconvertibleCharacters

On return, contains the number of characters in aForeign which were not converted. Characters which cannot be converted are output as Unicode replacement characters (0xFFFD).

TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter

On return, the index of the first byte of the first unconvertible character. For instance if the first character in the input descriptor (aForeign) could not be converted, then this parameter is set to the first byte of that character, i.e. zero. A negative value is returned if all the characters were converted.

TUint &aOutputConversionFlags

If the input descriptor ended in a truncated sequence, e.g. a multi-byte character that is not complete, aOutputConversionFlags returns with the EOutputConversionFlagInputIsTruncated flag set.

TUint aInputConversionFlags

Specify EInputConversionFlagAppend to append the converted text to aUnicode, otherwise the contents of aUnicode are overwritten. Specify EInputConversionFlagStopAtFirstUnconvertibleCharacter to stop converting when the first unconvertible character is reached. Specify EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable to prevent the function from returning the error-code EErrorIllFormedInput when the input descriptor consists of nothing but a truncated sequence.

Return value

TInt

The number of unconverted bytes left at the end of the input descriptor, or a negative error value defined in TError.


AsciiConversionData()

static IMPORT_C const SCnvConversionData &AsciiConversionData();

Description

Returns a ready-made SCnvConversionData object for converting between Unicode and ASCII. This can be passed into the aConversionData parameter to DoConvertFromUnicode() or DoConvertToUnicode().

Note: This utility function should only be called by a plug-in conversion library.

Return value

const SCnvConversionData &

ASCII conversion data object.


GetDowngradeForExoticLineTerminatingCharacters()

inline TDowngradeForExoticLineTerminatingCharacters GetDowngradeForExoticLineTerminatingCharacters();

Description

Return value

TDowngradeForExoticLineTerminatingCharacters

[Top]


Member classes


Class TArrayOfAscendingIndices

class TArrayOfAscendingIndices;

Description

Holds an ascending array of the indices of the characters in the source Unicode text which could not be converted by CCnvCharacterSetConverter::ConvertFromUnicode() into the foreign character set

Members

Defined in CCnvCharacterSetConverter::TArrayOfAscendingIndices:
AppendIndex(), EAppendFailed, EAppendSuccessful, NumberOfIndices(), Remove(), RemoveAll(), TAppendResult, TArrayOfAscendingIndices(), operator[]()

Construction and destruction


TArrayOfAscendingIndices()

inline TArrayOfAscendingIndices();

Description

C++ constructor. The array is initialised to be of length zero.

Member functions


AppendIndex()

IMPORT_C TAppendResult AppendIndex(TInt aIndex);

Description

Appends an index to the array of indices.

The value of aIndex should be greater than that of the last index in the array, to maintain an ascending array. The return value should be tested to see whether the function succeeded or not.

Parameters

TInt aIndex

The index to append to the array.

Return value

TAppendResult

EAppendFailed if the append failed, or EAppendSuccessful if it succeeded.


Remove()

inline void Remove(TInt aIndexOfIndex);

Description

Deletes a single index from the array.

Parameters

TInt aIndexOfIndex

The index of the index to delete. Must not be negative and must not be greater than the length of the array, or a panic occurs.


RemoveAll()

inline void RemoveAll();

Description

Deletes all indices from the array.


NumberOfIndices()

inline TInt NumberOfIndices() const;

Description

Returns the number of indices in the array.

Return value

TInt

The number of indices in the array.


operator[]()

inline TInt operator[](TInt aIndexOfIndex) const;

Description

Gets the value of the specified index.

Parameters

TInt aIndexOfIndex

Index into the array.

Return value

TInt

The value of the index.

Member enumerations


Enum TAppendResult

TAppendResult

Description

The return value of CCnvCharacterSetConverter::AppendIndex().

EAppendFailed

The append failed.

EAppendSuccessful

The append succeeded.

[Top]


Member structures


Struct SCharacterSet

struct SCharacterSet;

Description

Stores information about a non-Unicode character set. The information is used to locate the conversion information required by ConvertFromUnicode() and ConvertToUnicode().

An array of these structs that contain all available character sets can be generated by CreateArrayOfCharacterSetsAvailableLC() and CreateArrayOfCharacterSetsAvailableL(), and is used by one of the overloads of PrepareToConvertToOrFromL().

Members

Defined in CCnvCharacterSetConverter::SCharacterSet:
Identifier(), Name(), NameIsFileName()

Member functions


Identifier()

inline TUint Identifier() const;

Description

Gets the character sets UID.

Return value

TUint

The UID of the character set.


NameIsFileName()

inline TBool NameIsFileName() const;

Description

Tests whether a filename given by the function SCharacterSet::Name() is a real file name (i.e. conversion is provided by a plug in DLL), or just the character set name (i.e. conversion is built into Symbian OS).

Note: If the function returns ETrue then the path and filename can be parsed using TParse or TParsePtrC functions to obtain just the filename.

Return value

TBool

ETrue if the name is a real filename. EFalse if it is just the character set name.


Name()

inline TPtrC Name() const;

Description

Gets the full path and filename of the DLL which implements conversion for the character set.

If the character set is one for which conversion is built into Symbian OS rather than implemented by a plug in DLL, the function just returns the name of the character set. The NameIsFileName() function can be used to determine whether or not it is legal to create a TParsePtrC object over the descriptor returned by Name().

Notes:

The name returned cannot be treated as an Internet-standard name, it is locale-independent and should be mapped to the locale-dependent name by software at a higher level before being shown to the user. Conversion from Internet-standard names of character sets to the UID identifiers is provided by the member function ConvertStandardNameOfCharacterSetToIdentifierL().

Typically, to find the user-displayable name (as opposed to the internet-standard name) of a character set, you would do something like this:

        const CCnvCharacterSetConverter::SCharacterSet& characterSet=...;
        const TPtrC userDisplayable(characterSet.NameIsFileName()? TParsePtrC(characterSet.Name()).Name(): 
        characterSet.Name()); 

Return value

TPtrC

Full path and filename of the character set converter plug in DLL, or just the name of the character set.

[Top]


Member enumerations


Enum TAvailability

TAvailability

Description

Indicates whether a character set is available or unavailable  for conversion. Used by the second overload of PrepareToConvertToOrFromL().

EAvailable

The requested character set can be converted.

ENotAvailable

The requested character set cannot be converted.


Enum TError

TError

Description

Conversion error flags. At this stage there is only one error flag- others may be added in the future.

EErrorIllFormedInput

The input descriptor contains a single corrupt character. This might occur when the input descriptor only contains some of the bytes of a single multi-byte character.


Enum TEndianness

TEndianness

Description

Specifies the default endian-ness of the current character set. Used by SetDefaultEndiannessOfForeignCharacters().

ELittleEndian

The character set is big-endian.

EBigEndian

The character set is little-endian.


Enum TDowngradeForExoticLineTerminatingCharacters

TDowngradeForExoticLineTerminatingCharacters

Description

Downgrade for line and paragraph separators

EDowngradeExoticLineTerminatingCharactersToCarriageReturnLineFeed

Paragraph/line separators should be downgraded (if necessary) into carriage return and line feed pairs.

EDowngradeExoticLineTerminatingCharactersToJustLineFeed

Paragraph/line separators should be downgraded (if necessary) into a line feed only.


Enum anonymous

n/a

Description

Output flag used to indicate whether or not a character in the source descriptor is the first half of a surrogate pair, but is the last character in the descriptor to convert.

Note: This enumeration can be used in the DoConvertToUnicode() and DoConvertFromUnicode() functions. These are part of the Character Conversion Plug-in Provider API and are for use by plug-in conversion libraries only.

EInputConversionFlagAppend

Appends the converted text to the output descriptor.

EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable

By default, when the input descriptor passed to DoConvertFromUnicode() or DoConvertToUnicode() consists of nothing but a truncated sequence, the error-code EErrorIllFormedInput is returned. If this behaviour is undesirable, the input flag EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable should be set.

EInputConversionFlagStopAtFirstUnconvertibleCharacter

Stops converting when the first unconvertible character is reached.

EInputConversionFlagMustEndInDefaultCharacterSet

Appends the default character set Escape sequence at end of converted text

EInputConversionFlagAssumeStartInDefaultCharacterSet


Enum anonymous

n/a

Description

EOutputConversionFlagInputIsTruncated

Indicates whether or not the source descriptor ends in a truncated sequence, e.g. the first half only of a surrogate pair.


Enum anonymous

n/a

Description

Initial value for the state argument in a set of related calls to ConvertToUnicode().

KStateDefault


Enum anonymous

n/a

Description

ELowestThreshold

The lowest confidence value for a character set accepted by Autodetect