Class CBNFParser

class CBNFParser : public CBase, public MDataProviderObserver;

Description

Base class for parsers that use a BNF tree to parse an input stream.

The BNF tree form used is a variation on Extended BNF described in the XML1.0 specification. The general form of the tree is as follows:

Each node in the tree defines a rule that the input stream must meet to satisfy the grammar.

1. a node type is set to the rule type, as defined in TParserNodeTypes

2. node data stores any string required by the rule: e.g. for a comparison rule, the string to match against

3. the parser allows callback functions to be called either before or after the rule is processed. If these are present, they are stored as attributes of the node.

4. some rules allow sub-rules: for example, the AND rule expects a number of sub-rules, all of which must be successful if the AND rule itself is to succeed. Each sub-rule is represented as a child node of the parent rule. Sub-rules in turn can have sub-rules.

5. reference rule nodes are also allowed: these do not define themselves rules, but direct the parser to another rule. They can link rules to each other and so build rule sequences more complex than a simple tree.

All the top-level rules are stored as attributes of the root node. The attribute type is a string that names the rule; the attribute value is a pointer to the node that implements the rule.

The class supplies functions that encapsulate adding rules appropriately to the tree. The parser provider creates a derived class that implements the virtual method TreeL() that uses these functions to create a BNF rule tree.

The user of the parser initialises the parser with ResetL(), and then passes input data to the parser using ProcessData(). The parser supports partial parsing: the input stream does not have to completed before parsing can begin. As soon as data is added, the parser attempts to parse it.

Derivation

• MDataProviderObserver - MDataProviderObserver provides a virtual interface for any class to observe any MDataProvider, and provide data receiving services
• CBase - Base class for all classes to be instantiated on the heap

• CBNFParser - Base class for parsers that use a BNF tree to parse an input stream

Members

Defined in CBNFParser:
AddComponentAttributeL(), AddRuleCallbackL(), AndL(), AttributeLUT(), CBNFParser(), CRuleStack, CommitL(), CurrentRule(), DeleteMark(), EndConditional(), ExactL(), ExecutePostRuleCallbackL(), ExecutePreRuleCallbackL(), MDataProviderObserverReserved1(), MDataProviderObserverReserved2(), Mark(), MarkCallback(), MarkedL(), MarkedWithInitialTextL(), NMoreL(), NewBNFL(), NewComponentL(), NewComponentL(), NewComponentL(), NewComponentL(), NewComponentL(), NewComponentL(), NewL(), NewRuleL(), NewRuleL(), OptionalL(), OptionalMatched(), OrL(), ParseL(), PerformRuleL(), ProcessDataL(), RangeL(), ReferenceL(), ResetL(), RuleMatched(), SelectL(), SetBaseUriL(), SetDataExpected(), SetDocumentTypeL(), SetDocumentTypeL(), SetState(), SetStatus(), StartConditional(), State(), StringL(), TRuleCallback, TreeL(), Valid(), WithoutL(), iCurrentRule, iLUT, iMatched, iMoreCount, iMoreMaximum, iMoreMinimum, iOptionalMatched, iParsing, iPostRuleCallback, iPreRuleCallback, iRangeEnd, iRangeStart, iReferenceString, iRuleStack, iString, iStringComplete, iSubRule, iSubRuleMatched, iTree, ~CBNFParser()

Inherited from CBase:
Delete(), Extension_(), operator new()

static IMPORT_C CBNFParser *NewL(CAttributeLookupTable &aLUT);

Description

Allocates and constructs a new BNF parser.

Parameters

CAttributeLookupTable &aLUT Attribute lookup table in which to store attributes for the rule tree

Return value

CBNFParser * New parser

virtual IMPORT_C ~CBNFParser();

Description

Destructor.

protected: IMPORT_C CBNFParser(CAttributeLookupTable &aLUT);

Description

Constructor.

Parameters

CAttributeLookupTable &aLUT Attribute lookup table

virtual IMPORT_C void ResetL();

Description

Reset the parser to a state where it can accept and parse new input.

If no BNF tree yet exists the virtual method TreeL() is called to obtain the BNF tree for this parser. Any existing state of parsing and input data is destroyed.

inline TBool Valid() const;

Description

Checks if the input stream was completely processed

Return value

TBool ETrue if all of the data was processed, EFalse if the data didn't match to the parsing rules

inline HBufC *StringL() const;

Description

Concatenates the rest of the input stream (which hasn't yet been processed) into a single string. The ownership of the string is given to the caller.

Return value

HBufC * String containing the remaining data to be parsed. OWNERSHIP PASSED TO CALLED.

inline CBNFNode *CurrentRule();

Description

Gets a pointer to the rule node currently being processed.

Return value

CBNFNode * Rule node

inline void Mark();

Description

Set a mark to the current position of the input stream.

The mark acts as a tag in the stream currently being processed. As we process further along the stream after adding the mark, we can perform a rollback to the most previously set mark and start processing again (e.g. OR rule works this way). The string fragments won't be consumed (deleted) until all the marks on a fragment (and fragments before that) are deleted.

inline HBufC *MarkedL();

Description

Get string between the "cursor position" and the latest mark on the stream.

Return value

HBufC * Pointer to the string from the previous mark on to the current position of processed string. OWNERSHIP OF THE STRING GIVEN TO THE CALLER.

inline HBufC *MarkedWithInitialTextL(const TDesC &aInitialText);

Description

Gets the marked string with a string added before the mached string.

Parameters

const TDesC &aInitialText

Return value

HBufC * A string cosisting of aInitialText appended with the marked string. OWNERSHIP OF THE CONSTRUCTED STRING IS GIVEN TO THE CALLER.

See also:

• MarkedL()

inline void DeleteMark();

Description

Removes the latest mark. All the marks are stored in a stack and this removes the topmost mark.

inline TBool RuleMatched() const;

Description

Tests if the used rule matched.

This is typically used in post-rule callbacks.

Return value

TBool True if the used rule matched; otherwise false

inline TBool OptionalMatched() const;

Description

Tests if an Optional node sub-rule matched.

Return value

TBool True if the sub- rule matched; otherwise false

IMPORT_C CBNFNode *NewBNFL();

Description

Creates a new rule tree root node.

It creates a new single instance of CBNFNode as the root node of the rule tree. All the top-level rules are attached as attributes to this node. The root node should have single child node, which should be a reference to the logical root of the rule tree. This can be done be attaching the logical root rule as a component to the root rule.

Return value

CBNFNode * New rule tree root node

IMPORT_C CBNFNode &NewRuleL(CBNFNode *aRootRule, const TDesC &aRuleName, TParserNodeTypes aRuleType, HBufC *aData, TRuleCallback *aPreRule, TRuleCallback *aPostRule);

Description

Creates a new rule node and adds it to the root of the rule tree.

This overload takes ownership of the node data.

Parameters

CBNFNode *aRootRule Pointer to the root BNF node, created with NewBNFL() const TDesC &aRuleName Reference to a string identifying this rule. The string is used to make references to this rule from other rule's subtrees. TParserNodeTypes aRuleType Rule type HBufC *aData Rule data pointer. This is used with EExact and ESelect type rules to match actual text strings. TRuleCallback *aPreRule Function pointer to a pre-rule function that gets called before the parser starts processing this rule and its children (i.e. the rule subtree). TRuleCallback *aPostRule Function pointer to a post-rule function that gets called after the parser has processed this rule and its subtree.

Return value

CBNFNode & The newly created rule node in the rule tree

IMPORT_C CBNFNode &NewRuleL(CBNFNode *aRootRule, const TDesC &aRuleName, TParserNodeTypes aRuleType, const TDesC &aData, TRuleCallback *aPreRule, TRuleCallback *aPostRule);

Description

Creates a new rule node and adds it to the root of the rule tree.

This overload takes a reference to the node data instead of owning it.

Parameters

CBNFNode *aRootRule Pointer to the root BNF node, created with NewBNFL() const TDesC &aRuleName Reference to a string identifying this rule. The string is used to make references to this rule from other rule's subtrees. TParserNodeTypes aRuleType Rule type const TDesC &aData Rule data pointer. This is used with EExact and ESelect type rules to match actual text strings. TRuleCallback *aPreRule Function pointer to a pre-rule function that gets called before the parser starts processing this rule and its children (i.e. the rule subtree). TRuleCallback *aPostRule Function pointer to a post-rule function that gets called after the parser has processed this rule and its subtree.

Return value

CBNFNode & The newly created rule node in the rule tree

IMPORT_C CBNFNode *NewComponentL(TParserNodeTypes aRuleType, const TDesC &aData);

Description

Creates a new rule node, but does not add it to the tree.

This overload sets no rule callbacks.

Parameters

TParserNodeTypes aRuleType Rule type const TDesC &aData Rule data reference. This is used with EExact and ESelect type rules to match actual text strings.

Return value

CBNFNode * The new rule node

IMPORT_C CBNFNode *NewComponentL(TParserNodeTypes aRuleType, HBufC *aData=0, TRuleCallback *aPreRule=0, TRuleCallback *aPostRule=0);

Description

Creates a new rule node, but does not add it to the tree.

This overload allows rule callbacks to be set.

Parameters

TParserNodeTypes aRuleType Rule type HBufC *aData Rule data pointer. This is used with EExact and ESelect type rules to match actual text strings. TRuleCallback *aPreRule Function pointer to a pre-rule function that gets called before the parser starts processing this rule and its subtree. TRuleCallback *aPostRule The new rule node

Return value

CBNFNode * The new rule node

IMPORT_C CBNFNode *NewComponentL(CBNFNode *aRootRule, const TDesC &aRuleName);

Description

Creates a new reference rule node.

Parameters

CBNFNode *aRootRule Root node const TDesC &aRuleName Rule name

Return value

CBNFNode * The new reference rule node

IMPORT_C CBNFNode &NewComponentL(CBNFNode &aParentRule, TParserNodeTypes aRuleType, const TDesC &aData);

Description

Creates a new sub-rule, and makes it a child of a specified parent rule.

This overload sets no rule callbacks.

Parameters

CBNFNode &aParentRule The rule to the new sub-rule shall be added as a child TParserNodeTypes aRuleType Rule type const TDesC &aData Rule data reference. This is used with EExact and ESelect type rules to match actual text strings.

Return value

CBNFNode & The new sub-rule node

IMPORT_C CBNFNode &NewComponentL(CBNFNode &aParentRule, TParserNodeTypes aRuleType, HBufC *aData=0, TRuleCallback *aPreRule=0, TRuleCallback *aPostRule=0);

Description

Creates a new sub-rule, and makes it a child of a specified parent rule.

This overload sets rule callbacks.

Parameters

CBNFNode &aParentRule The rule to the new sub-rule shall be added as a child TParserNodeTypes aRuleType Rule type HBufC *aData Rule data pointer. This is used with EExact and ESelect type rules to match actual text strings. TRuleCallback *aPreRule Function pointer to a pre-rule function that gets called before the parser starts processing this rule and its subtree. TRuleCallback *aPostRule Function pointer to a post-rule function that gets called after the parser has processed this rule and its subtree.

Return value

CBNFNode & The new sub-rule node

IMPORT_C CBNFNode &NewComponentL(CBNFNode *aRootRule, CBNFNode &aParentRule, const TDesC &aRuleName);

Description

Creates a new reference rule node, and adds it as a child of the specified parent.

Note that the function succeeds even if the target rule aRuleName does not yet exist.

Parameters

CBNFNode *aRootRule Root node: the function needs this to find the target rule identified by aRuleName CBNFNode &aParentRule Parent rule to which to attach the new rule as a child const TDesC &aRuleName The name of the rule that the new node refers to

Return value

CBNFNode & The new reference rule node

IMPORT_C void AddComponentAttributeL(CBNFNode &aRule, CBNFNodeAttributeType aAttribute, TInt aInt);

Description

Adds an additional attribute to an existing rule node.

For example, this is used with range rules, which specify the range boundaries using start and end attributes.

Parameters

CBNFNode &aRule Rule node on which to set the attribute CBNFNodeAttributeType aAttribute Attribute type TInt aInt Attribute value

virtual IMPORT_C void ProcessDataL(HBufC8 &aData);

Description

Called by the data provider to add data for the parser to process.

This implements MDataProviderObserver::ProcessDataL().

Parameters

HBufC8 &aData The data to process

virtual IMPORT_C void SetStatus(TInt aStatus=KErrNone);

Description

Called by the data provider to report its status to its observer.

This implements MDataProviderObserver::SetStatus().

Parameters

TInt aStatus Status code

virtual IMPORT_C void SetDocumentTypeL(const TDesC &);

Description

Intended Usage : The request to construct a CDataHandler to process data of aDataType, where aDataType is a string to match against some plugins declared data handling ability.

Parameters

const TDesC &

virtual IMPORT_C void SetDocumentTypeL(const TDesC &, const TDesC &);

Description

Intended Usage : The request to construct a CDataHandler for aDataType with non default document encoding where aDataType is a string to match against some plugins declared data handling ability.

Parameters

const TDesC & const TDesC &

virtual IMPORT_C void SetDataExpected(TInt);

Description

Intended Usage : The report of how many bytes of data is expected to be passed to ProcessDataL, enabling age reporting capabilities.

Parameters

TInt

virtual IMPORT_C void SetBaseUriL(const TDesC *aBaseUri);

Description

Intended Usage : Set the URI that the DataProvider considers the base URI for the current data.

Parameters

const TDesC *aBaseUri A const pointer to the uri

virtual IMPORT_C void MDataProviderObserverReserved1();

Description

Intended Usage : Reserved for future expansion.

virtual IMPORT_C void MDataProviderObserverReserved2();

Description

Intended Usage : Reserved for future expansion.

IMPORT_C void CommitL();

Description

Notifies the parser that all the data has been passed in.

It causes the parser to parse any of the input stream not already parsed.

inline TParseState State() const;

Description

Get the current state of the parser.

Return value

TParseState Parser state

protected: virtual IMPORT_C TBool ReferenceL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes a reference rule node.

It is called by PerformRuleL().

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool ExactL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an EExact rule node.

It is called by PerformRuleL().

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool RangeL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an ERange rule node.

It is called by PerformRuleL().

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool SelectL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an ESelect rule node.

It is called by PerformRuleL().

if (aMatched == CFragmentedString::EMatch)

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool WithoutL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an EWithout rule node.

It is called by PerformRuleL().

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool AndL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an EAnd rule node.

It is called by PerformRuleL().

performedMatch = ETrue;

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool OrL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an EOr rule node.

It is called by PerformRuleL().

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C TBool OptionalL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Parameters

CBNFNode &aRule CFragmentedString::TStringMatch &aMatched

Return value

TBool

protected: virtual IMPORT_C TBool NMoreL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Processes an ENMore rule node.

It is called by PerformRuleL().

Parameters

CBNFNode &aRule The rule node being processed CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: virtual IMPORT_C void AddRuleCallbackL(CBNFNode &aRule, const TDesC *aCallbackID, TRuleCallback *aCallback);

Description

Adds a callback to a rule.

Parameters

CBNFNode &aRule The rule to which the callback is to be added const TDesC *aCallbackID Callback type: either CBNFNode::KPreRuleCallback() or CBNFNode::KPostRuleCallback() TRuleCallback *aCallback Callback function

protected: virtual IMPORT_C void ExecutePreRuleCallbackL(CBNFNode &aRule);

Description

Executes a pre-rule callback function.

Parameters

CBNFNode &aRule Node specifying the callback

protected: virtual IMPORT_C void ExecutePostRuleCallbackL(CBNFNode &aRule);

Description

Executes a post-rule callback function.

Parameters

CBNFNode &aRule Node specifying the callback

protected: virtual IMPORT_C CBNFNode *TreeL();

Description

Return value

CBNFNode *

protected: virtual IMPORT_C void StartConditional(TParserNodeTypes aRuleType);

Description

Parameters

TParserNodeTypes aRuleType

protected: virtual IMPORT_C void EndConditional(TParserNodeTypes aRuleType, TBool aSuccess);

Description

Parameters

TParserNodeTypes aRuleType TBool aSuccess

protected: static IMPORT_C void MarkCallback(CBNFParser &aParser);

Description

Inserts a mark to the current position of the stream being processed.

Adding a mark is a very common callback operation before starting to process a rule, so the method is provided by the parser.

Parameters

CBNFParser &aParser Parser processing the stream

protected: IMPORT_C CAttributeLookupTable &AttributeLUT() const;

Description

Gets the attribute look-up table used by this parser.

Return value

CAttributeLookupTable & Attribute look-up table

protected: IMPORT_C TBool ParseL();

Description

Return value

TBool

protected: virtual IMPORT_C TBool PerformRuleL(CBNFNode &aRule, CFragmentedString::TStringMatch &aMatched);

Description

Handles a node in the rule tree.

It calls the appropriate handler method for the rule type.

Parameters

CBNFNode &aRule Rule node CFragmentedString::TStringMatch &aMatched On return, flag indicating if input stream matched the rule

Return value

TBool ETrue if the function completed processing this node, otherwise EFalse

protected: inline void SetState(TParseState aState);

Description

Sets the parser state.

Parameters

TParseState aState Parser state

protected: typedef CStack<CBNFNode, EFalse> CBNFParser::CRuleStack;

Description

Defines a type to handle a stack of rules.

protected: typedef void( CBNFParser::TRuleCallback)(CBNFParser&);

Description

Type definition for a callback function pointer Callback functions need to get a reference to the parser as parameter and they need to be static.

protected: CAttributeLookupTable & iLUT;

Description

Storage object for all the attributes and identifiers in a tree

protected: CFragmentedString iString;

Description

An utility object which stores all the buffers passed into the parser and represents them as if they would form a single, continuous string. This class also performs the actual physical matching/selection of the strings and holds the marks set onto the string.

protected: TBool iStringComplete;

Description

Flag indicating if the input stream has been completely processed.

protected: CBNFNode * iTree;

Description

The BNF tree the parser is using to parse the input stream.

protected: CRuleStack iRuleStack;

Description

A stack of rules from iTree which are waiting to be completed. The stack basically holds the path along the rule tree.

protected: CBNFNode * iCurrentRule;

Description

The BNF rule that is currently being processed.

protected: TBool iSubRuleMatched;

Description

Flag that indicates when returning to a rule in the rulestack if the child rule matched correctly.

protected: TBool iOptionalMatched;

Description

Flag that indicates when returning to a rule in the rulestack if an optional rule matched correctly.

protected: CBNFNode * iSubRule;

Description

The child rule we are returning from (if any). If this is NULL we are new to this BNF rule.

protected: TParseState iParsing;

Description

Parser state.

protected: CFragmentedString::TStringMatch iMatched;

Description

Input stream matched rule flag.

protected: const TDesC * iReferenceString;

Description

Stores attribute identifier for reference string attributes.

protected: const TDesC * iRangeStart;

Description

Stores attribute identifier for range start attributes.

protected: const TDesC * iRangeEnd;

Description

Stores attribute identifier for range end attributes.

protected: const TDesC * iMoreMinimum;

Description

Stores attribute identifier for nmore minimum attributes.

protected: const TDesC * iMoreCount;

Description

Stores attribute identifier for nmore count attributes.

protected: const TDesC * iMoreMaximum;

Description

Stores attribute identifier for nmore maximum attributes.

protected: const TDesC * iPreRuleCallback;

Description

Stores attribute identifier for pre-rule callback attributes.

protected: const TDesC * iPostRuleCallback;

Description

Stores attribute identifier for post-rule callback attributes.