String pools

• Motivation

• String pool

• Key features

• Static string tables

• Building a static string table

• Using the string pool

Many systems, components and applications have to deal with pre-defined, well known string constants. For example, parsing and manipulating text that has structure almost always requires comparisons against standard string constants.

In a complex system, composed of a large number of objects, there may also be a need to pass strings between objects, and to route processing depending on the value of some string. The implementation of the HTTP transport framework and the XML framework are examples within Symbian OS where such intense string handling is required.

To improve efficiency, Symbian OS uses the idea of the string pool.

A string pool is a mechanism for storing strings in such a way that makes the comparison of strings a very fast operation. It is particularly efficient at handling strings that can be set up at program compile time, for example the kind of strings that identify lexical elements in a structured text. Typically, they are well known strings that are likely to be used very often in a given context.

Such strings are organised into tables, and each string within a table can be referenced by an index value, which can be symbolised by an enum. Such tables are referred to as static string tables. The basic algorithm used internally ensures that the pool only contains one string of any particular value, and uses a reference counting mechanism to keep track of usage.

The advantages of representing string constants in such a way are:

• avoiding a proliferation of duplicate strings throughout a component or an application; typically there is one string pool per thread, and one copy of a string

• allowing string constants to be represented by integer values

• allowing strings to be passed between objects by passing integer values, wrapped in a class (one of the classes RString and RStringF)

• allowing strings to be compared by simply comparing the integer values.

Internally, a string pool uses hash tables to reference strings.

Static string tables and string constants can be added dynamically to the string pool, for example, at run time. However, there is always a performance penalty when adding either a static or a dynamic string to the string pool as the hash tables need to be updated. This means that it is better to add static string tables at string pool initialisation time, as this is the best time to absorb the overhead.

A string pool is referenced through an RStringPool, a handle like object. The following diagram shows the basic idea:

• The string pool as supplied by Symbian OS supports any strings that can be represented by a TDes8 descriptor; this includes ASCII or UTF-8 encoded strings. Note that a TDes8 type can represent any type of data , including binary data, and means that a string pool can easily handle the extended characters of another encoding.

• Within the string pool, strings are of two types: case sensitive and case insensitive. This affects the way strings are compared. Case insensitivity implies that strings are folded for the purpose of comparison.

• A string pool can contain up to 4,096 static string tables, and each table can represent up 26,2144 strings.

Static string tables are defined and built at compile time. They are represented by a TStringTable object. You add a string table to the string pool by passing the string table reference to a call to:

void RStringTable::OpenL(const TStringTable& aTable);

The following diagram shows the general picture. Note that the strings in any given string table are deemed to be either case sensitive or case insensitive, and this governs how comparisons are made.

As the name implies, a static string table is declared as a const TStringTable data member of a class whose name you are free to choose. The class name is defined in a header file while the table itself is implemented in a .cpp C++ source file. Both the header file and the C++ source file are normally included in the project definition. Typically, a set of enum values are also defined within the scope of the class, and each value is associated with the strings in the table; code in other parts of the program access the strings in the string pool using these enum values.

The Perl script, stringtable.pl, located in ...\syslibs\bafl\stringtools\, can be used to generate these .cpp and .h files from a simple text definition. The text definition file simply lists the strings and the enum symbols to be associated with them; the file itself is given a .st file type.

The file ExampleStringTable.st is a simple example:

# Example String Table
fstringtable ExampleStringTable
!// Some types of fruit
# This comment won't appear in the .h file, but the one above will.
EApple apple
EOrange orange
EBanana banana
# Some animals
ECat cat
EDog dog

The main points to note are:

• the keyword fstringtable is used to define the name of the class that contains the string table declaration and the enum value symbols. The class name itself follows the keyword - here it is ExampleStringTable.

  Note that you can include underscore characters in the class name, for example: Example_StringTable.

• the symbols EApple and EOrange form the enum value symbols that correspond to the strings apple and orange respectively.

• all statements starting with a # are comments and are completely ignored. However # characters can appear in a string. For example ap#ple is a valid string and is not interpreted as a comment.

• all statements starting with a ! are comments that are inserted into the generated header file.

Running the Perl script with ExampleStringTable.st as source generates the header file ExampleStringTable.h and the C++ source file ExampleStringTable.cpp as shown below:

// Autogenerated from epoc32\build\generated\example\ExampleStringTable.st by the stringtable tool - Do not edit

#ifndef STRINGTABLE_ExampleStringTable
#define STRINGTABLE_ExampleStringTable

#include "StringPool.h"

struct TStringTable;

/** A String table */
class ExampleStringTable 
    {
public:
    enum TStrings
        {
        // Some types of fruit
        /** apple */
        EApple,
        /** orange */
        EOrange,
        /** banana */
        EBanana,
        /** cat */
        ECat,
        /** dog */
        EDog
        };
    static const TStringTable Table; 
    };

#endif // STRINGTABLE_ExampleStringTable

// Autogenerated from epoc32\build\generated\example\ExampleStringTable.st by the stringtable tool - Do not edit
#include <e32std.h>
#include "StringPool.h"
#include "StringTableSupport.h"
#include "ExampleStringTable.h"
#ifdef _DEBUG
#undef _DEBUG
#endif

_STLIT8(K1, "apple");
_STLIT8(K2, "orange");
_STLIT8(K3, "banana");
_STLIT8(K4, "cat");
_STLIT8(K5, "dog");

// Intermediate
const void * const KStringPointers[] =
    {
    (const void*)&K1,
    (const void*)&K2,
    (const void*)&K3,
    (const void*)&K4,
    (const void*)&K5
    };

const TStringTable ExampleStringTable::Table = {5, KStringPointers, EFalse};

The table itself is the static data member Table of class ExampleStringTable.

We'll use this example to show you how to use string pools.

A static string table can be built in any .mmp file; the most common use is as part of building an application.

1. Add the following lines to the .mmp file:

   START STRINGTABLE ExampleStringTable.st
EXPORTPATH  /epoc32/include
END

   This code:

   • generates the .cpp file and the .h file

   • copies the generated .h file to epoc/include

   • handles the generated .cpp file as part of the source of the overall executable.

2. Include the following in your bld.inf file:

   PRJ_MMPFILES
.\StringTableExample.mmp

The following is an example .mmp file, StringTableExample.mmp:

// StringTableExample.MMP
//
// Copyright (c) Symbian Ltd 2001
//

TARGET          StringTableExample.exe
TARGETTYPE      exe

SYSTEMINCLUDE   \epoc32\include

SOURCEPATH .
SOURCE StringTableExample.cpp

START STRINGTABLE ExampleStringTable.st
EXPORTPATH  /epoc32/include
END

LIBRARY EUSER.LIB BAFL.LIB
VENDORID 0x70000001

Note: The previous example .mmp file builds the following small .cpp file that uses the string pool:

// StringTableExample.cpp
//
// Copyright (c) Symbian Software Ltd, 2004-2007. All rights reserved.
//

#include "e32base.h"
#include "e32svr.h"
#include "StringPool.h"
#include "ExampleStringTable.h"

void StartL()
    {
    TBuf<100> buf;
    RStringPool pool;
    pool.OpenL(ExampleStringTable::Table);
    buf.Copy(pool.StringF(ExampleStringTable::EApple,ExampleStringTable::Table).DesC());
    RDebug::Print(buf);
    }

// main loop
//
GLDEF_C TInt E32Main()
    {
    // Install exception handler
    CTrapCleanup* theCleanup = CTrapCleanup::New(); 
    TRAPD(err,StartL());
    delete theCleanup;
    return(KErrNone);
    }

• Adding tables to the string pool

• Accessing strings

• Comparing strings

• Retrieving the string itself

• You can create a string pool with no static string tables. This is, in effect an empty string pool. Static string tables can be added later. Use the following variant of OpenL():

  void RStringPool::OpenL()

• You can add a static string table to the pool by using the following variant of OpenL().

  void RStringPool::OpenL(const TStringTable& aTable)

  This function creates the string pool, if it doesn't already exist, before adding the string table. Use the same function to add further string tables whenever you need to.

  You can use this same variant to add more static string tables at a later time.

• There is a variation on the OpenL() function that provides a mechanism that notifies you when the string pool is being closed. This is the variant:

  void OpenL(const TStringTable& aTable, MStringPoolCloseCallBack& aCallBack);

  Closing the string pool is done using the RStringPool::Close() function; this closes all open references to the string pool. This may be important in environments where you use asynchronous programming techniques. You provide an implementation of the MStringPoolCloseCallBack to handle the situation.

• Note that the implementation assumes that the string tables remain in existence and are located at the same address for as long as the string pool remains open. This assumption is almost always justified as the tables are static data.

Accessing strings involves getting a handle to a string in the string pool. These handles are instances of RString and RStringF classes, and they are what your code uses to represent strings.

RString represents a string that is case-sensitive, for example, a string that is compared in a case-sensitive manner.

RStringF represents a string that is case-insensitive, for example, a string that is compared in a case-insensitive manner. For example, when you come to compare two RStringF objects where the represented strings only differ in terms of case, then the two strings are considered identical.

There are a number of ways of creating an RString to represent a specific string. Note: Creating an RStringF object is the same as creating an RString. Where RString is discussed, this information also applies to RStringF:

• create an RString to represent a specific string in a static string table by using the variant:

  RString RStringPool::String(TInt aIndex,const TStringTable& aTable) const;

  passing the reference to the specific string table, and the index number that identifies the string. The following code, based on the example string table above, is a commonly used pattern to generate an RString. Here we generate an RString to represent the string "banana". The code does no error handling and ignores leave situations.

  #include <>

RStringPool thepool;
RString thebanana;
...
thepool.OpenL(ExampleStringTable::Table);
thebanana = thepool.String(ExampleStringTable::EBanana,ExampleStringTable::Table);
...

  In more sophisticated code, and where there is more than one static string table, you might use a function or a member function of some class to retrieve that TStringTable reference. The following is a common pattern:

  void X::Foo(CSomeClass aInstance,...)
    {
    ...
    thebanana = thepool.String(ExampleStringTable::EBanana,aInstance.GetTable());
    ...
    }

  where GetTable() is a function that returns a reference to a specific TStringTable.

• create an RString to represent a string that may not exist in any existing static string table within the string pool. You use the function:

  RString  RStringPool::OpenStringL(const TDesC8& aString) const;

  This is an example of dynamically adding a string to the pool (which also generates the RString to represent it). If the string already exists in a static string table, then the string value itself is not added; instead a new reference to the existing string is created. The following is a simple code fragment:

  class X
    {
public :
    ...
    void Foo(const TDesC8& aString,...);
    ...
    }

  void X::Foo(aString,...)
    {
    ...
    RString theString;
    theString = thepool.OpenStringL(aString);
    ...
    }

Comparing strings becomes a fast and simple process once an RString handle (or an RStringF) has been created. A common use is to route processing based on the value of the string, for example:

class X
    {
public :
    ...
    void Foo(RString aString,...);
    ...
    }

void X::Foo(aString,...)
    {
    ...
    switch (aString)
        {
    case ExampleStringTable::EBanana :
        {
        // do something
        }
    case ExampleStringTable::EOrange :
        {
        // do something
        }
    ...
    }

Given an RString object (or an RStringF), it is always possible to retrieve the string value itself. To do this use the DesC() function that is provided by the base class RStringBase of the classes RString and RStringF.

For example:

class X
    {
public :
    ...
    void Foo(RString aString, ...);
    ...
    }

void X::Foo(aString)
    {
    const TDesC8& stringValue = aString.DesC();
    ...
    }