1.1���Limitations

The parser is not complete. There are grammatical constructions that it does not process. I hope to fill it out later. In the mean time, because PLY makes it reasonably easy to add new grammar rules, it should be fairly easy to extend the parser to cover constructions that you need. (A few notes on how to do this are included below.) If you add new rules, please pass them back to me. If you do not believe that you can add them yourself, please send me a request.

1.2���Requirements

• You will need Python, of course.
• You will need to install PLY, which is available at: http://systems.cs.uchicago.edu/ply/.

2.1���Where to get it

A distribution file containing the source, a few sample compact syntax files that the parser has been tested on, and this documentation is at: relaxngcompact-1.0a.tar.gz.

2.2���Installation

Use the standard Python Disutils installation. Run:

python setup.py build
python setup.py install     # as root

2.3���Testing the installation

Use the run_tests.py script in the tests sub-directory to test the parser. For example:

cd tests
python run_tests.py

3.1���Running the parser

1. From the command line -- Examples:

   • Get help:

     $ python relaxngcompact.py --help
     usage:
         relaxngcompact.py [options] <infile.rng>
     example:
         relaxngcompact.py infile.rng
     
     Parse RELAX NG compact syntax.
     options:
       --version    show program's version number and exit
       -h, --help   show this help message and exit
       -d, --debug  run lexer and parser in debug mode
     

   • Parse a file and show the parse tree:

     $ python relaxngcompact.py test6.rng
     

2. From within Python code -- Here is an example:

   import sys
   import relaxngcompact
   
   tree = relaxngcompact.parse('test5.rng')
   tree.show(sys.stdout, 0)
   

3.2���Using the parse tree

The show method in class ASTNode provides an example which shows how to walk the parse tree:

#
# Representation of a node in the AST (abstract syntax tree).
#
class ASTNode:
    o
    o
    o
    def show(self, outstream, level):
        self.show_level(outstream, level)
        if self.children:
            childCount = len(self.children)
        else:
            childCount = 0
        if self.value is None:
            value = 'None'
        else:
            value = '"%s"' % self.value
        # Show the node type and value (if there is one).
        outstream.write('Type: %s  Value: %s  Child #: %d\n' % (
            NodeType2NameDict[self.nodeType],
            value,
            childCount))
        # Show the children.
        for child in self.children:
            child.show(outstream, level + 1)

    def show_level(self, outstream, level):
        for idx in range(level):
            outstream.write('  ')

Here are a few additional notes:

• Constants are provided for the various node types, for example:

  NodeType_elementpattern =            1
  NodeType_textpattern =               2
  NodeType_pattern =                   3
      o
      o
      o
  

• There is a dictionary (NodeType2NameDict) that maps the node type constants to descriptive names. For example:

  >>> import relaxngcompact
  >>> relaxngcompact.NodeType2NameDict[relaxngcompact.NodeType_textpattern]
  'TextPattern'
  

• Class ASTNode provides access to the following attributes for each node in the parse tree:

  • getType(): Returns the node type (e.g. one of NodeType_elementpattern, NodeType_textpattern, ...).
  • getValue(): Returns the value of the node. If the node contains text, then the value will contain this string.
  • getChildren(): Returns a list of children (instances of ASTNode).

4.1���Adding a new AST node type

In order to add a new AST node type, do the following:

1. Add a node type constant. For example:

   NodeType_newnodename = 36
   

2. Add an element to the NodeType2NameDict dictionary. For example:

   NodeType_newnodename: 'NewNodeName',
   

4.2���Adding a new token type

In order to enable the lexer to recognize a new token (type), do the following:

1. Add a string to the tuple relaxngcompact.tokens. This is the name that you will use in your grammatical function to refer to the token.

2. Define the token:

   • If the token is punctionation, then add a constant with a name beginning with "t_". For example:

     t_PERIOD = r'\.'
     

   • If the token is a keyword, then add a new element to the relaxngcompact.reserved dictionary. For example:

     'newtoken': 'NEWTOKEN',
     

     where:

     • newtoken is the string to be recognized by the lexer.
     • NEWTOKEN is the name that will be used to refer to the token in grammar rules/functions.

4.3���Adding a new grammar rule

In order to add a new grammar rule that recognizes a new grammatical expression, do the following:

1. Add a new AST node type to represent the expression. See above.

2. If the rule must recognize any new tokens not already defined, then add new token types. See above.

3. Add a new parsing function to implement the grammar rule. Here is an example:

   def p_newrulename(t):
       'newrulename : NEWTOKEN other_rule_1 COMMA other_rule_2'
       node = ASTNode(NodeType_newrule, t[2], t[4])
       t[0] = node
   

   Explanation:

   • The function name must begin with "p_".
   • The doc string for the function contains the grammar rule.
   • In the body of the function/rule, t[1], t[2], ... refer to the first, second, ... components of the matched expression.
   • The rule/function returns the node that it constructs by assigning a value to t[0].

4. If the rule must recognize any complex, nested expressions (for example other_rule1 and other_rule_2, in our example), and these are not already defined, then add grammar rules for those expressions.