======================================== A Quixote Application Skeleton Generator ======================================== :author: Dave Kuhlman :address: dkuhlman@rexx.com http://www.rexx.com/~dkuhlman :revision: 1.3b :date: Aug. 16, 2004 :copyright: Copyright (c) 2004 Dave Kuhlman. This documentation and the software it describes is covered by The MIT License: http://www.opensource.org/licenses/mit-license. :abstract: ``quixote_appgen`` is a Python application that generates a skeleton of a Quixote application. ``quixote_appgen`` reads an application specification from a configuration file, which is a Python module containing a Python data structure. ``quixote_appgen`` generates source code for a Quixote application to which you can "easily" add application specific code. .. sectnum:: :depth: 4 .. contents:: :depth: 4 Introduction ============ ``quixote_appgen`` generates the skeleton of a Quixote application. It follows a specific directory structure, which is described below. This directory structure is the same one followed by `A Quixote Application Skeleton`_. .. _`A Quixote Application Skeleton`: http://www.rexx.com/~dkuhlman/quixote_skeleton.html In addition, the configuration file can be used to specify a variety of options, for example: - The name of the application and the application directory. - The name of the user interface directory. - The name of the services directory. - The names of the modules in the user interface directory. These PTL modules implement the user interface. - The names of classes in the user interface. - The names of methods in each class in the interface. - The names of the modules in the services directory/package. - The names of classes in each module in the services package. - The names of methods in each class in each module in the services package. Why you might care ------------------ Here is what ``quixote_appgen`` may be able to do for you: - Generate the skeleton of a Quixote application. This may enable you to generate the top level request processing code without knowing a great deal about Quixote programming. And, reading the generated code may help you learn how to do some of these Quixote tasks. - Enforce a consistent request processing mechanism across the top level of your Quixote application by generating consistent code. Doing so may make your application and its structure easier to understand than one which does ad hoc processing for each request. In effect, ``quixote_appgen`` can generate the skeleton of the implementation of your Quixote application and can encourage/enforce the top level structure of your application while leaving you to fill in domain and business logic. Another way to say this might be to say that ``quixote_appgen`` is about top-level mechanism, not policy. In other words ``quixote_appgen`` generates and, thus, encourages or enforces a specific consistent top-level mechanism, while leaving the implementation of capabilities, business logic, business objects, etc to the user-developer. In fact, it is a goal of ``quixote_appgen`` that the user-developer be able to plug in existing class libraries, function libraries, business components, etc. ``quixote_appgen`` also attempts to provide a small amount of support for the development cycle *after* the initial generation of code. Here are several things it does in this regard: - ``quixote_appgen`` generates separate "implementation" files. These are modules that you can maintain separately from the files ``quixote_appgen`` generates to implement the top level mechanism for your application. This enables you to add and edit domain specific code, and yet later re-generate the top level mechanism code without over-writing your edited code. - ``quixote_appgen`` can generate source code that uses the Python unit testing framework to perform tests on your application. It also generates "implementation" files for unit tests, where you can place code that tailors individual tests to your needs. And, ``quixote_appgen`` generates simple support for REST (REpresentational State Transfer). See `Example -- REST-ful dispatching`_ for more on this. Additional help --------------- ``quixote_appgen`` is all about constructing and setting up a Quixote application. You will find additional help and suggestions for this task in `A Quixote Application Skeleton`_. .. _`A Quixote Application Skeleton`: http://www.rexx.com/~dkuhlman/quixote_skeleton.html Where to Get ``quixote_appgen`` =============================== ``quixote_appgen`` is available here: http://www.rexx.com/~dkuhlman/quixote_appgen-1.3b.tar.gz. How to Install ``quixote_appgen`` ================================= You will need to run ``quixote_appgen.py`` as a script. So, place ``quixote_appgen.py`` where you can conveniently run it. You can also run the ``setup.py`` script to install ``quixote_appgen`` in the standard Python location. This may be useful if you intend to write your own Python script that imports and calls functions in ``quixote_appgen``. To do so, run the following:: $ python setup.py build $ python setup.py install # as root How to Use ``quixote_appgen`` ============================= In general, you will follow these steps: 1. Create a configuration file -- Additional guidance is below. 2. Run ``quixote_appgen`` -- Change to the directory under which you wish to generate your application. For example:: cd myrootdir python quixote_appgen.py -a -i -u myappspecfile.xml This will generate your application (application files, implementation files, and unit test files) in a directory whose name is specified as "dir_name" in the specification file ``myappspecfile.xml`` immediately under the directory myrootdir. Creating a specification file ----------------------------- The specification file is an XML document. There is a sample in the distribution named ``genapp-config.xml``. As an alternative, the specification file can also be a Python module that contains a data structure named "application". This data structure is composed of dictionaries and lists that define the application to be generated. In order to use this form of specification file, you need to employ the command line flag ``--python-spec``. The discription that follows describes the Python code form of specification file. I'll add description of the XML form later. However, the structure of these two forms is very similar and even the names are very close: usually the names in the XML document have a dash where the Python names have an underscore. So most of what follows should help with the XML form, also. Loosely speaking, the structure of the XML document and the Python object follows the structure of the directories (Python packages), modules, and classes that it generates. For example, this object contains a "scripts" sub-object containing specifications of the files generated in the ``scripts`` directory. You can create a specification by copying one of the templates in the distribution and then using your text editor to copy, paste, and edit. The template files contain comments to help guide you while making modifications. There is an XML template or example (genapp-config.xml) and there are two Python templates: (1) a fully nested one (spec_template.py) and (2) one in which the specification data structure is slightly unrolled (spec_template_unrolled.py). Take your pick. There is also a sample (slightly unrolled) in the distribution: test_spec_unrolled.py. Each of the Python format samples and templates contains a simple "main" function that pretty prints the specification data structure. That gives you an easy way to view your specification as a whole, whether it is unrolled or not. The remainder of this section describes the application specification data structure. This structure is basically the same for both the Python form and the XML form. The example specification files should make this reasonably clear. The top level data structure is a dictionary stored in a variable named ``application``. This dictionary contains the following keys and values: - "dir_name" -- (a string) The name of the application and the directory in which the application is to be generated. - "scripts" -- (a dictionary) Describes the scripts directory. This dictionary contains the following keys and values: - "dir_name" -- (a string) The name of the scripts directory. - "servers" -- (a list of strings) The servers for which start-up scripts are to be generated. Currently, ``quixote_appgen`` is capable of generating scripts for SCGI and Medusa servers. Therefore, the possible values are: "scgi" and "medusa". - "ui" -- (a dictionary) Describes the user interface directory. This dictionary contains the following: - "dir_name" -- (a string) The name of the user interface directory. - "modules" -- (a list of dictionaries) Each dictionary describes a Python module in the user interface directory. Each dictionary contains the following keys and values: - "name" -- (a string) -- The name of the Python module to be generated. - "classes" -- (a list of dictionaries) Each dictionary describes one of the classes in the module. Each dictionary contains the following keys and values: - "name" -- (a string) The name of the class. - "methods" -- (a list of strings) The names of the methods in the class. - "services" -- (a dictionary) This dictionary describes the directory (a Python package) that contains implementation of services called from the user interface package. This dictionary contains the following keys and values: - "dir_name" -- (a string) The name of the services directory. - "modules" -- (a list of dictionaries) Each dictionary describes a Python module in the services directory. Each dictionary contains the following keys and values: - "name" -- (a string) -- The name of the Python module to be generated. - "classes" -- (a list of dictionaries) Each dictionary describes one of the classes in the module. Each dictionary contains the following keys and values: - "name" -- (a string) The name of the class. - "methods" -- (a list of strings) The names of the methods in the class. Running ``quixote_appgen`` -------------------------- ``quixote_appgen`` takes a single argument: the name of the Python module containing the application specification. When you run ``quixote_appgen``, it will create a subdirectory in the directory from which you run it. The name of the subdirectory is the application name specified in the configuration file. Make sure that the directory from which you run ``quixote_appgen`` does not already contain a directory by that name. Here is an example showing how to run ``quixote_appgen``. Suppose (1) that I want to generate my application under directory ``myquixotedirectory`` and (2) that my specification is stored in ``myspecification.xml``. Then the following will generate application files (-a), implementation files (-i), and unit test files (-u):: $ cd myquixotedirectory $ python quixote_appgen.py -a -i -u myspecification.xml Or, using the Python form of specification:: $ cd myquixotedirectory $ python quixote_appgen.py -a -i -u --python-spec myspecification Command line flags ------------------ ``quixote_appgen`` also responds to a variety of command line flags. Here is the usage information:: usage: python quixote_appgen.py [options] example: python quixote_appgen.py myapplicationspec.xml options: -h, --help show this help message and exit -a, --application generate application (scripts, ui, and service) modules -i, --implementation generate implementation modules in services and ui directories -u, --unittests generate unit tests in the /tests directory -s, --scripts generate scripts in the /scripts directory --python-spec read specification from python code, not XML -v, --verbose show verbose info during generation -pPAGER, --pager=PAGER pager for showing differences etc -d, --dry-run dry-run/simulate; do not actually write files --force-all force (no ask) over-writing all files --backup-all backup before over-writing all files --coupling-style=COUPLING_STYLE coupling style for implementation classes --help-coupling-style display possible values for option coupling-style -rRUN_TIME_ENGINE, --run-time-engine=RUN_TIME_ENGINE insert run-time engine --force-appl force (no ask) over-writing of application files --backup-appl backup/age application files before over-writing --maxversions-appl=MAXVERSIONS_APPL maximum versions of application files --rest-priority=REST_PRIORITY priority of processing by application code ('plain- before-rest'/'rest-before-plain') --force-impl force (no ask) over-writing of implementation files --backup-impl backup/age implementation files before over-writing --maxversions-impl=MAXVERSIONS_IMPL maximum versions of implementation files --force-unit force (no ask) over-writing of unit test files --backup-unit backup/age unit test files before over-writing --maxversions-unit=MAXVERSIONS_UNIT maximum versions of unit test files --force-scripts force (no ask) over-writing of script (directory) files --backup-scripts backup/age script (directory) files before over-writing --maxversions-scripts=MAXVERSIONS_SCRIPTS maximum versions of script (directory) files Flags specifying operations are: a, application Generate application modules. This flag tells ``quixote_appgen`` to generate source code for the core application. i, implementation Generate implementation modules. This flag tells ``quixote_appgen`` to generate modules containing stubs for the methods called from the generated methods in the user interface and services directory. These methods are those for (1) pre-condition, (2) task, and (3) post-condition. u, unittests Generate unit tests. This flag tells ``quixote_appgen`` to generate source code for unit tests. Unit tests are generated under the application directory in a subdirectory named ``tests``. s, scripts Generate files in the scripts directory. This flag tells ``quixote_appgen`` to generate the driver scripts and the default configuration file in the scripts directory. You must specify at least one of "-a", "-i", "-u", or "-s" flags (or the equivalent long form). Flags with effect across application, implementation, unit testing: v, verbose Display the names of the files that are created. d, dry-run Just write out the names of the files that would be created. Do *not* actually create the files. force-all Force over-writing of all files without asking. Configuration file: section "general", option "force". backup-appl Backup and age all files before over-writing. Configuration file: section "general", option "backup". coupling-style ``quixote_appgen`` can generate several different styles of coupling (1) between user interface classes and user interface implementation classes, (2) between service classes and service implementation classes, and (3) between unit test classes and unit test implementation classes. These coupling styles are: inheritance Generate code which makes the implementation class a super-class and call methods in the super-class. class-delegation Generate code in which the constructor of the base class is passed an implementation *class*, which it instantiates, then saves the instance. The base class calls methods in the implementation class through this instance. instance-delegation Generate code in which the constructor of the base class is passed an *instance* of an implementation class, which it saves. The base class calls methods in the implementation class through this instance. peak Generate code that uses PEAK component binding to enable the base class to call methods in the implementation class. help-coupling-style Display helpful information about the possible values of the ``coupling-style`` option. Here is that usage information:: coupling-style-help: possible values for option --coupling-style: "inheritance" -- inheritance; implementation class is super-class "class-delegation" -- standard Python delegation; ctor takes a class "instance-delegation" -- standard Python delegation; ctor takes an instance "peak" -- generate PEAK-style linkage r, run-time-engine Insert boilerplate for a run-time engine into the ``__init__.py`` in the user interface directory. You should provide, as the value of this flag, the path/name of a file containing the boilerplate code. The ``quixote_appgen`` distribution contains a sample file ``rte1.py``. See section `The Run-time Engine`_ for more on this. Flags for application related options are: force-appl Force over-writing of application files without asking. Configuration file: section "application", option "force". backup-appl Backup and age application files before over-writing. Configuration file: section "application", option "backup". maxversions-appl The maximum number of versions of application files to create and maintain while aging files during backup. Configuration file: section "application", option "maxversions". rest-priority The priority or precedence of processing by generated application code. This option determines whether the generated code will first attempt to process the request as a REST request or as a normal request. The possible values are ``plain-before-rest`` and ``rest-before-plain``. Configuration file: section "implementation", option "rest-priority". Flags for implementation related options are: force-impl Force over-writing of implementation files without asking. Configuration file: section "implementation", option "force". backup-impl Backup and age implementation files before over-writing. Configuration file: section "implementation", option "backup". maxversions-impl The maximum number of versions of implementation files to create and maintain while aging files during backup. Configuration file: section "implementation", option "maxversions". Flags for unit test related options are: force-unit Force over-writing of unit test files without asking. Configuration file: section "unittests", option "force". backup-unit Backup and age unit test files before over-writing. Configuration file: section "unittests", option "backup". maxversions-unit The maximum number of versions of unit test files to create and maintain while aging files during backup. Configuration file: section "unittests", option "maxversions". Flags for script directory related options are: force-scripts Force over-writing of files in the scripts directory without asking. Configuration file: section "scripts", option "force". backup-scripts Backup and age unit files in the scripts directory before over-writing. Configuration file: section "scripts", option "backup". maxversions-scripts The maximum number of versions of files in the scripts directory to create and maintain while aging files during backup. Configuration file: section "unittests", option "maxversions". And, if neither the force flag nor the backup flag are used, when a file already exists, the following menu of options are presented:: File Mynewapp/Services/moduleAImpl.py exists. (b)ackup (or age) files, then write. (r)eplace existing file. (s)kip this file. Do not generate. show (d)ifferences and ask again. name (n)ew version. (q)uit. Enter option: (b/r/s/d/n/m/q): Explanation: 1. *(b)ackup* does, for this file only, what the ``--backup-xxxx`` flag does for all files in a category. The existing files are aged and renamed to backup versions before the current file is written. 2. *(r)eplace* over-writes the current file. It is the same as the ``--force-xxxx`` flag applied to this file only. 3. *(s)kip* prevents writing this file. This file is not written. 4. *(d)ifferences* displays the differences between the file to be generated and the existing (old) version of the file. The command ``diff --unified=4`` is used to generate the differences. The pager program used to display the differences is a configuration option; the default is ``less``. 5. *(n)ew* asks for a new name, then writes a file with this name. 6. *(q)uit* exits ``quixote_appgen`` immediately. Aging -- Files are aged to files named X.py.1, X.py.2, X.py.3, ..., from the youngest to the oldest. The maximum number of backup versions is a configuration option. In effect, for implementation file X.py, the following renaming is performed:: X.py.2 --> X.py.3 X.py.1 --> X.py.2 X.py --> X.py.1 Configuration files ------------------- Many of the options described above can be specified in a configuration file. ``quixote_appgen`` looks for configuration files in the following locations and in the following order: 1. /etc/quixote_appgen.conf 2. $HOME/.quixote_appgenrc 3. ./quixote_appgen.conf Options found in configuration files *lower* in this list are merged with and override those of files *higher* in the list. For example, if the ``backup`` option for application code is found in both ``$HOME/.quixote_appgenrc`` and ``./quixote_appgen.conf``, then the value in ``quixote_appgen.conf`` in the current directory overrides that from ``.quixote_appgenrc`` in the user's home directory. Here is a sample configuration file:: [general] verbose = 1 dry-run = 0 pager = less #force-all = true #backup-all = true #coupling-style = inheritance coupling-style = class-delegation #coupling-style = instance-delegation #coupling-style = peak #run-time-engine = rte1.py [application] force = 0 backup = 0 maxversions = 3 #rest-priority = plain-before-rest rest-priority = rest-before-plain [implementation] force = 0 backup = 0 maxversions = 3 [unittests] force = 0 backup = 0 maxversions = 3 [scripts] force = 0 backup = 0 maxversions = 3 Explanation: - The configuration file is parsed and processed with the ``ConfigParser`` module from the standard Python library. See `5.15 ConfigParser -- Configuration file parser`_ for more details. - The ``verbose`` flag tells ``quixote_appgen`` to display the names of created files. - ``pager`` in the ``general`` section is the program used to display certain results, for example, differences between current and previous versions of generated files. If not specified, the value of the environment variable ``PAGER`` is used; and, if that is not specified, the default is ``less`` (an improved version of ``more``). - The remaining options are described in the section `Command line flags`_, above. .. _`5.15 ConfigParser -- Configuration file parser`: http://docs.python.org/lib/module-ConfigParser.html An example ---------- There is an example in the sub-directory ``Examples`` of the distribution. This example contains a specification file which, when used with ``quixote_appgen``, will generate a small application. You can run this example by doing the following:: cd Examples # Generate application files python quixote_appgen.py -a --python-spec test_spec_unrolled # Generate implementation files python quixote_appgen.py -i --python-spec test_spec_unrolled # Generate unit test files python quixote_appgen.py -u --python-spec test_spec_unrolled # Generate files in the scripts directory. python quixote_appgen.py -s --python-spec test_spec_unrolled Or, all in one and with verbose output:: python quixote_appgen.py -a -i -u -s -v --python-spec test_spec_unrolled And, here is the same thing, this time using an XML specification file:: python ../quixote_appgen.py -a -i -u -s -v genapp-config.xml Implementation Files ==================== ``quixote_appgen`` supports "implementation" files. These files provide a separation between user-developer edited code and generated code. This applies to both the services and user interface directories. For option ``coupling-style=inheritance``, it works something like this: - Each module ``X`` generated in the user interface and services directory imports a module ``XImpl``. - Each class ``Y`` in module ``X`` in the services directory inherits (sub-classes) class ``YImpl`` in module ``XImpl``. - Each class ``Y`` in module ``X`` in the services directory calls methods in class ``YImpl`` in module ``XImpl``. - ``quixote_appgen`` can generate skeleton files containing empty versions of the needed classes and their methods. - Then during later phases of development, ``quixote_appgen`` can do either of the following: 1. Not (re-)generate implementation files, thus preserving user-developer edits. If you added new classes and methods to the services directory, then you will have to edit and add these new classes and methods to the corresponding implementation file. (Omit command line flag ``-i``.) 2. Generate new implementation files, but before doing so, age and backup existing implementation files. The user-developer must then merge the newly generated version and the edited version. (Use command line flags ``-i`` and ``-b``.) See `Running quixote_appgen`_ for an explanation of these and other options. Something analogous is generated for other values of option ``coupling-style`` (``class-delegation``, ``instance-delegation``, and ``peak``). You can run the example in the ``Example`` directory with different values of option ``coupling-style`` in order to see the code generated for each style. In the services directory, the generated code looks something like the following for the different values of ``coupling-style``: - ``coupling-style=inheritance``:: class Myclass1(moduleAImpl.Myclass1Impl): def serviceA11(self, *args): if not self.serviceA11_pre_condition_check(): return False self.serviceA11_task() if not self.serviceA11_post_condition_check(): return False return True - ``coupling-style=class-delegation``:: class Myclass1: def __init__(self, impl_class=moduleAImpl.Myclass1Impl): self.impl_instance = impl_class() def serviceA11(self, *args): if not self.impl_instanceserviceA11_pre_condition_check(): return False self.impl_instance.serviceA11_task() if not self.impl_instance.serviceA11_post_condition_check(): return False return True - ``coupling-style=instance-delegation``:: class Myclass1: def __init__(self, impl_instance=None): if impl_instance is None: self.impl_instance = moduleAImpl.Myclass1Impl() else: self.impl_instance = impl_instance def serviceA11(self, *args): if not self.impl_instance.serviceA11_pre_condition_check(): return False self.impl_instance.serviceA11_task() if not self.impl_instance.serviceA11_post_condition_check(): return False return True - ``coupling-style=peak``:: class Myclass1(binding.Component): def __init__(self, impl_class=moduleAImpl.Myclass1Impl): Myclass1.impl_component = binding.Make(impl_class, attrName='impl_component') def serviceA11(self, *args): if not Myclass1.impl_component.serviceA11_pre_condition_check(): return False Myclass1.impl_component.serviceA11_task() if not Myclass1.impl_component.serviceA11_post_condition_check(): return False return True Analogous code is generated in the user interface directory. How to Modify ``quixote_appgen`` ================================ You can, of course, modify ``quixote_appgen`` so that it generates code tailored to your needs. This section explains several examples of modifying ``quixote_appgen``, although in some cases these modifications have already been incorporated into the distribution. Example -- REST-ful dispatching ------------------------------- **Note** -- The modification described in this section has been implemented in the currect version of ``quixote_appgen``. This section describes a modification developed and suggested by gian paolo ciceri (gp.ciceri@acm.org). The modification makes the generated code more compliant with REST (REpresentational State Transfer). REST is "an architectural style for large-scale software design". It is particularlly appropriate for Web applications. And, Quixote is especially helpful in implementing REST-ful Web applications. See the following for more information on REST: http://rest.blueoxen.net/cgi-bin/wiki.pl. This modification generates UIHandler methods in the ``__init__.py`` module in the UI directory which check for the existence of an ID test method in the generated class. If that method exists, it is called to determine whether the current component of the URI is acceptable, and, if so, traversal is continued on the next component of the URI. Here is a sample of the generated code:: class AuthorUIHandler: _q_exports = [] def _q_lookup(self, request, name): klass = AuthorUI() meth = getattr(klass, 'do_' + name, None) # try to see if there's an available object id checker objectIdChecker = getattr(klass, 'checkObjectId', None) if meth: return meth(request) elif objectIdChecker: # try to check if the given QUERY_PATH element is a valid object Id if klass.checkObjectId(name): # add the object id to the request object with a well-known form name request.form['_q_objectId'] = name # repeat the traversal to the next step handler = AuthorUIHandler() return handler else: # the path element is not a valid object Id, complain handler = DefaultUI() return handler.index(request) else: handler = DefaultUI() return handler.index(request) The intent of this modification is to generate code that responds to URIs of the form:: http://.../AuthorUI/WilliamFaulkner/getBirthPlace http://.../AuthorUI/WilliamFaulkner/getTitleList instead of:: http://.../AuthorUI/getBirthPlace?authorId=WilliamFaulkner http://.../AuthorUI/getTitleList?authorId=WilliamFaulkner That is, from the end user's (REST-ful) point of view, one can request several operations (``getBirthPlace``, ``getTitleList``, ...) on a single resource (in this case ``WilliamFaulkner``). The generated code processes these requests by first checking the resource name (in this case ``WilliamFaulkner``) before traversing the URI to invoke an operation on that resource (``getBirthPlace``, ``getTitleList``, ...). A little additional explanation: - The generated code performs its extra steps (check the ID and traverse further if accepted) only if the UI class contains a method ``checkObjectId``. If that method is not present, then processing is the same as that generated by the original ``quixote_appgen``. - The generated code performs its extra steps (check the ID and traverse further if accepted) only if the UI class does *not* contain a method named "do\_" + name. That is, normal processing takes precedence over REST processing. Note the ``if-elif`` statement. It is easy to imagine a change to ``quixote_appgen`` which reverses this order. Also see command line flag "--rest-priority". - The method ``checkObjectId`` takes a single argument (the current component of the URI) and returns True or False. - If method ``checkObjectId`` returns False, then the generated code does something to indicate an error. In this case, it merely re-displays the default UI page. And, here is a replacement for function ``gen_ui_init_uihandler`` in ``quixote_appgen`` that generates this REST-ful code. You will also find the following code in ``quixote_appgen``, where it is commented out. Update: the REST-ful version is now active, and the older version is commented out:: def gen_ui_init_uihandler(add, uiModules): for module in uiModules: for klass in module['classes']: klassName = klass['name'] add('class %sHandler:' % klassName) add(' _q_exports = []') add(' def _q_lookup(self, request, name):') add(' klass = %s()' % klassName) add(' meth = getattr(klass, \'do_\' + name, None)') add(' # If there is an object id checker, retrieve it.') add(' objectIdChecker = getattr(klass, \'checkObjectId\', None)') add(' if meth:') add(' return meth(request)') add(' elif objectIdChecker:') add(' # Is the given QUERY_PATH element is a valid object Id?') add(' if klass.checkObjectId(name):') add(' # Add the object id to the request object with') add(' # a well-known form name.') add(' request.form[\'_q_objectId\'] = name') add(' # Now, repeat the traversal to the next step.') add(' handler = %sHandler()' % klassName) add(' return handler') add(' else:') add(' # The path element is not a valid object Id, so complain.') add(' handler = DefaultUI()') add(' return handler.index(request)') add(' else:') add(' handler = DefaultUI()') add(' return handler.index(request)') add('') add('') Example -- pre and post conditions ---------------------------------- This modification inserts calls to pre and post condition check methods in each method in the user service directory, thus providing a small amount of support for the Design by Contract methodology. (For information on Design by Contract, see: http://c2.com/cgi/wiki?DesignByContract.) Here is a sample class:: import moduleBImpl class Myclass4(moduleBImpl.Myclass4Impl): def serviceB41(self, *args): if not self.serviceB41_pre_condition_check(): return False self.serviceB41_task() if not self.serviceB41_post_condition_check(): return False return True In addition, ``quixote_appgen`` now accepts a flag on the command line ("-i" or "--implementation") and, if present, generates a separate module containing method skeletons for the pre and post condition check methods. Putting these in separate files which are only generated when that flag is present enables the user-developer to edit and add code that is not over-written every time ``quixote_appgen`` is run. Here is a sample implementation class from a generated implementation file:: class Myclass4Impl: def serviceB41_pre_condition_check(self, *args): return True def serviceB41_post_condition_check(self, *args): return True def serviceB41_task(self, *args): pass Example -- *coupling-style* options ----------------------------------- In this section we'll go through the steps I followed in adding a small amount of support for several styles of coupling between the generated application classes and the implementation classes. This extension enables the user to select one of the following styles of coupling: - *inheritance* -- The original and still the default. - *class-delegation* -- Call methods in the implementation class through a class that is passed in to the constructor. The default class is the implementation class generated by ``quixote_appgen``. - *instance-delegation* -- Call methods in the implementation class through an instance that is passed in to the constructor. The default instance is an instance of the implementation class generated by ``quixote_appgen``. - *peak* -- Call methods in the implementation class through a PEAK feature exposed by the implementation class. Summary -- In order to add this new capability to ``quixote_appgen``, we'll follow these steps: 1. Set the default option for *coupling-style* in the configuration options structure. 2. Add code that reads a *coupling-style* from the ``quixote_appgen`` configuration file. 3. Add a *coupling-style* option to the command line parser. 4. Check for a *coupling-style* command line option. 5. Implement generation of each of the different coupling styles in the user interface directory. 6. Implement generation of each of the different coupling styles in the services directory. Set the default *coupling-style* option ....................................... ``GEN_CONFIG_OPTS`` is a global structure that contains (mostly) options from the configuration file and the command line. It is initialized to default values. We add a default for *coupling-style*:: GEN_CONFIG_OPTS = ConfigurationOptions() o o o GEN_CONFIG_OPTS.CouplingStyle = 'inheritance' Read the *coupling-style* configuration option .............................................. We'll place the *coupling-style* option in the ``general`` section of the configuration file, since it applies to both the user interface directory and the services directory. We add code to function ``read_configuration_options`` that will check the configuration parser for a *coupling-style* option in the ``general`` section:: if parser.has_option(sectionName, 'coupling-style'): GEN_CONFIG_OPTS.EnablePeak = parser.get(sectionName, 'coupling-style') Add a *coupling-style* option to the command line parser ........................................................ In function ``populate_option_parser``, we add an ``--coupling-style`` to the instance of OptionParser that ``quixote_appgen`` has created:: parser.add_option("--coupling-style", type="string", dest="coupling_style", help="coupling style for implementation classes") Check for the *coupling-style* command line option .................................................. We test and save the command line option for PEAK in function ``get_commandline_options``:: if options.coupling_style: GEN_CONFIG_OPTS.CouplingStyle = options.coupling_style Generate coupling styles in the user interface directory ........................................................ In order add the generation of the new coupling code in the user interface directory, we modify function ``gen_ui_ui_klass``. The new function checks the *coupling-style* option, then calls one of several new functions based on the value of that option. Here is the modified function ``gen_ui_ui_klass`` and the new functions that it calls:: # # Generate one class in a ui module in the user interface directory. # def gen_ui_ui_klass(options, add, moduleName, klass): if GEN_CONFIG_OPTS.CouplingStyle == 'inheritance': gen_ui_ui_klass_inheritance(options, add, moduleName, klass) elif GEN_CONFIG_OPTS.CouplingStyle == 'class-delegation': gen_ui_ui_klass_class_delegation(options, add, moduleName, klass) elif GEN_CONFIG_OPTS.CouplingStyle == 'instance-delegation': gen_ui_ui_klass_instance_delegation(options, add, moduleName, klass) elif GEN_CONFIG_OPTS.CouplingStyle == 'peak': gen_ui_ui_klass_peak(options, add, moduleName, klass) def gen_ui_ui_klass_inheritance(options, add, moduleName, klass): klassName = klass['name'] add('class %s(%sImpl.%sImpl):' % (klassName, moduleName, klassName)) methodNames = options.get_list_by_key(klass, 'methods') generatedMethod = 0 for methodName in klass['methods']: urlPath = "%s/%s/%s" % (moduleName, klassName, methodName) methodPath = "%s.%s.%s" % (moduleName, klassName, methodName) add(' def do_%s [html] (self, request):' % methodName) add(' header("%s")' % urlPath) add(' self.%s_generate_content(request)' % methodName) add(' footer()') add('') generatedMethod = 1 if not generatedMethod: add(' pass') add('') def gen_ui_ui_klass_class_delegation(options, add, moduleName, klass): klassName = klass['name'] add('class %s:' % klassName) add('') add(' def __init__(self, impl_class=%sImpl.%sImpl):' % (moduleName, klassName)) add(' self.impl_instance = impl_class()') add('') methodNames = options.get_list_by_key(klass, 'methods') generatedMethod = 0 for methodName in klass['methods']: urlPath = "%s/%s/%s" % (moduleName, klassName, methodName) methodPath = "%s.%s.%s" % (moduleName, klassName, methodName) add(' def do_%s [html] (self, request):' % methodName) add(' header("%s")' % urlPath) add(' self.impl_instance.%s_generate_content(request)' % methodName) add(' footer()') add('') generatedMethod = 1 if not generatedMethod: add(' pass') add('') def gen_ui_ui_klass_instance_delegation(options, add, moduleName, klass): klassName = klass['name'] add('class %s:' % klassName) add('') add(' def __init__(self, impl_instance=None):') add(' if impl_instance is None:') add(' self.impl_instance = %sImpl.%sImpl()' % (moduleName, klassName)) add(' else:') add(' self.impl_instance = impl_instance') add('') methodNames = options.get_list_by_key(klass, 'methods') generatedMethod = 0 for methodName in klass['methods']: urlPath = "%s/%s/%s" % (moduleName, klassName, methodName) methodPath = "%s.%s.%s" % (moduleName, klassName, methodName) add(' def do_%s [html] (self, request):' % methodName) add(' header("%s")' % urlPath) add(' self.impl_instance.%s_generate_content(request)' % methodName) add(' footer()') add('') generatedMethod = 1 if not generatedMethod: add(' pass') add('') def gen_ui_ui_klass_peak(options, add, moduleName, klass): klassName = klass['name'] add('class %s(binding.Component):' % (klassName, )) add('') add(' def __init__(self):') add(" %s.impl_component = binding.Make(%sImpl, attrName='impl_component')" % (klassName, klassName)) add('') methodNames = options.get_list_by_key(klass, 'methods') generatedMethod = 0 for methodName in klass['methods']: urlPath = "%s/%s/%s" % (moduleName, klassName, methodName) methodPath = "%s.%s.%s" % (moduleName, klassName, methodName) add(' def do_%s [html] (self, request):' % methodName) add(' header("%s")' % urlPath) add(' %s.impl_component.%s_generate_content(request)' % (klassName, methodName)) add(' footer()') add('') generatedMethod = 1 if not generatedMethod: add(' pass') add('') Generate coupling styles in the services directory .................................................. In order add the generation of the new coupling code in the services directory, we modify function ``gen_services_klass``. The new function checks the *coupling-style* option, then calls one of several new functions based on the value of that option. Here is the modified function ``gen_services_klass`` and the new functions that it calls:: # # Generate one class in an application module in the services directory. # def gen_services_klass(options, add, moduleName, klass): if GEN_CONFIG_OPTS.CouplingStyle == 'inheritance': gen_services_klass_inheritance(options, add, moduleName, klass) elif GEN_CONFIG_OPTS.CouplingStyle == 'class-delegation': gen_services_klass_class_delegation(options, add, moduleName, klass) elif GEN_CONFIG_OPTS.CouplingStyle == 'instance-delegation': gen_services_klass_instance_delegation(options, add, moduleName, klass) elif GEN_CONFIG_OPTS.CouplingStyle == 'peak': gen_services_klass_peak(options, add, moduleName, klass) def gen_services_klass_inheritance(options, add, moduleName, klass): klassName = klass['name'] add('class %s(%sImpl.%sImpl):' % (klassName, moduleName, klassName)) methodGenerated = 0 key = '%s.%s' % (moduleName, klassName) methodNames = options.get_list_by_key(klass, 'methods') for methodName in methodNames: add(' def %s(self, *args):' % methodName) add(' if not self.%s_pre_condition_check():' % methodName) add(' return False') add(' self.%s_task()' % methodName) add(' if not self.%s_post_condition_check():' % methodName) add(' return False') add(' return True') add('') methodGenerated = 1 if not methodGenerated: add(' pass') add('') def gen_services_klass_class_delegation(options, add, moduleName, klass): klassName = klass['name'] add('class %s:' % klassName) add('') add(' def __init__(self, impl_class=%sImpl.%sImpl):' % (moduleName, klassName)) add(' self.impl_instance = impl_class()') add('') methodGenerated = 0 key = '%s.%s' % (moduleName, klassName) methodNames = options.get_list_by_key(klass, 'methods') for methodName in methodNames: add(' def %s(self, *args):' % methodName) add(' if not self.impl_instance%s_pre_condition_check():' % methodName) add(' return False') add(' self.impl_instance.%s_task()' % methodName) add(' if not self.impl_instance.%s_post_condition_check():' % methodName) add(' return False') add(' return True') add('') methodGenerated = 1 if not methodGenerated: add(' pass') add('') def gen_services_klass_instance_delegation(options, add, moduleName, klass): klassName = klass['name'] add('class %s:' % klassName) add('') add(' def __init__(self, impl_instance=None):') add(' if impl_instance is None:') add(' self.impl_instance = %sImpl.%sImpl()' % (moduleName, klassName)) add(' else:') add(' self.impl_instance = impl_instance') add('') methodGenerated = 0 key = '%s.%s' % (moduleName, klassName) methodNames = options.get_list_by_key(klass, 'methods') for methodName in methodNames: add(' def %s(self, *args):' % methodName) add(' if not self.impl_instance.%s_pre_condition_check():' % methodName) add(' return False') add(' self.impl_instance.%s_task()' % methodName) add(' if not self.impl_instance.%s_post_condition_check():' % methodName) add(' return False') add(' return True') add('') methodGenerated = 1 if not methodGenerated: add(' pass') add('') def gen_services_klass_peak(options, add, moduleName, klass): klassName = klass['name'] add('class %s(binding.Component):' % klassName) add('') add(' def __init__(self, impl_class=%sImpl.%sImpl):' % (moduleName, klassName)) add(" %s.impl_component = binding.Make(impl_class, attrName='impl_component')" % (klassName, )) add('') methodGenerated = 0 key = '%s.%s' % (moduleName, klassName) methodNames = options.get_list_by_key(klass, 'methods') for methodName in methodNames: add(' def %s(self, *args):' % methodName) add(' if not %s.impl_component.%s_pre_condition_check():' % (klassName, methodName)) add(' return False') add(' %s.impl_component.%s_task()' % (klassName, methodName)) add(' if not %s.impl_component.%s_post_condition_check():' % (klassName, methodName)) add(' return False') add(' return True') add('') methodGenerated = 1 if not methodGenerated: add(' pass') add('') The Run-time Engine =================== The facility described above generates Python code that implements a "static" mapping from URLs to methods (actually to a triple: a method in a class in a module). This section describes an alternative approach: the run-time engine. What the run-time engine does ----------------------------- The run-time engine is a chunk of boiler plate code that is inserted into the application that you generate with ``quixote_appgen``, in particular, it is inserted in ``__init__.py`` in the user interface directory. The run-time engine looks at each component in the request path, and compares the component with corresponding components in the rules specified in the deployment configuration file. Note that the run-time engine is boiler plate Python code. It is available as a Python source code file which you can copy and modify. You may want to consider replacing the run-time engine with one tailored to your specific needs. How the run-time engine works ----------------------------- This section describes the implementation of the run-time engine that is included in the distribution. This implementation is in file ``rte1.py`` and is supported by code in ``runtime_config.py`` and ``runtime_config_sub.py``. 1. For each request, the run-time engine creates an instance of class ``Resolver``, then calls the ``_q_lookup`` method in that instance. 2. The ``_q_lookup`` method of class ``Resolver`` processes one component of the request path. It does the following: 1. Check the component against each action/rule in the deployment configuration file. For each rule that fails, mark that rule as inactive (that is, eliminate the rule from consideration for the current request). 2. For each component except the last (right-most), return myself (self: the instance of class Resolver), to be used for further traversal of components in the path. 3. When the last (right-most) component is reached, apply the first of any remaining (active) rules. The run-time engine described above uses modules generated by ``generateDS.py`` to parse and give access to the deployment configuration file. These files and a sample deployment configuration file are in the ``Deployment`` directory in the distribution: - ``Deployment/runtime-config.xml`` -- An example deployment configuration XML document. - ``Deployment/runtime_config.py`` -- Data bindings for the deployment configuration XML document type. Contains access classes for the XML elements in the deployment configuration XML document. Also contains a parser and support functions, for example, export methods. - ``Deployment/runtime_config_sub.py`` -- Subclasses of the data binding classes. Additional support specific to the run-time engine has been added to these sub-classes. How to generate the run-time engine ----------------------------------- Use the "-r" or "--run-time-engine" flag, giving it the name of a file containing boilerplate. This inserts the run-time engine into the file ``__init__.py`` in the user interface directory. An example implementation of a run-time engine is provided in ``rte1.py`` in the distribution. So, for example, to use the default run-time engine, you might run the following (adding the path to rte1.py, if necessary):: python quixote_appgen.py -a -r rte1.py genapp-config.xml The deployment configuration file --------------------------------- The deployment configuration file is an XML document that describes mappings from request paths to the methods that are to be called to satisfy the requests. A separate rule is specified for each mapping. More than one rule can specify the mapping to the same method. The distribution contains two Python modules that support access to the deployment configuration file. (This code generated with ``generateDS.py``. See http://www.rexx.com/~dkuhlman/generateDS.html.) These modules are: - ``runtime_config.py`` -- Classes for parsing the XML deployment file and for accessing its elements. - ``runtime_config_sub.py`` -- Sub-classes that add methods specific to the run-time engine. The deployment configuration document type is defined by the XSchema document ``runtime-config.xsd``. The elements in the XMLBehavior document type are the following: - -- The base element in the document. - -- Information global to the application. - -- The base URL used to access the application. - -- The connection string used to connect to your relational database. Depending on the implementation of the Python DB API that you use, you will need either or in order to access your database. - The parameters used to connection to your relational database. These include: - -- The host name. - -- The connection port, if using TCP. - -- The database. - -- The user name. - -- The user's database password. - -- Information about unit tests for the application. - -- The directory containing unit test code, possibly generated by ``quixote_appgen``. - -- Rules used to map request paths to the method to be called to satisfy the request. As each component in the actual request is considered, from left to right, rules may be eliminated. When all components in the actual path have been considered, the first of any surviving rules is applied, that is its element is evaluated. If no rules survive, the is applied. - -- A rule that describes a mapping from a path to a method. The path is described by a pattern that must match the request path. - -- A request path, which must match the request in order for this rule to be applied. The path is given as a sequence of regular expressions separated by forward slashes, for example: "/Myapplication/viewdata/id01003". Alternatively, the path can be described as a (see below). - -- A sequence of descriptions of the components in the request path. This rule matches only if every component in the sequence matches. - -- One component description for each component in the request path. - -- A regular expression that must match the component in order for this rule to survive for consideration. - -- A method to be called to validate the component. This action/rule will survive for consideration only if this method returns true. The method should take one parameter: the component. - -- The method to which the request is forwarded if this action matches the path. - -- A default rule to be applied if no survives. - -- The method to be evaluated if no matches. The value should be of the form "module_name.class_name.method_name" (at least, for ``rte1.py``). Unit Tests ========== ``quixote_appgen`` can generate code for performing unit tests. The generated code uses the Python unit testing framework (module ``unittest`` from the Python standard library). See `5.3 unittest -- Unit testing framework`_ from the ``Python Library Reference`` for more information. .. _`5.3 unittest -- Unit testing framework`: http://docs.python.org/lib/module-unittest.html Use the ``-u`` or ``--unittests`` flag to direct ``quixote_appgen`` to generate code for unit tests. What the unit tests do ---------------------- The generated unit test code is capable of performing the following operations: 1. Down-load and save the content from the responses to each request in the application, one file per request. 2. Down-load the response to each request in the application. Compare the response to the request to the content stored in the corresponding file. The code that performs the actual test is in a file named ``tests/testsImpl.py``. You can edit and modify this file to perform more complex tests than the simple comparison generated by ``quixote_appgen``. **Note:** The coupling style for the generated code is controlled by the same flag that controls the coupling style for code generated in the services directory. See `Command line flags`_. Down-load and save ------------------ Use either of the following to down-load the current responses:: python tests.py -s python tests.py -s -d Data Explanation: - The ``-s`` flag tells the unit test application to down-load and save the content. - The ``-d`` flag is used to specify a directory (other than the current directory) into which the content files are stored. Running the unit tests ---------------------- Use one of the following (or something like therm) to run the unit tests:: python tests.py python tests.py -d Data python tests.py -u "http://localhost:8081/" -d Data Explanation: - The ``-d`` flag is used to specify a directory (other than the current directory) into which the content files are stored. Unit tests command line flags ----------------------------- Here is the usage information from a sample unit test file:: usage: python tests.py [options] example: python tests.py -d Data python tests.py -s -u "http://localhost:8081/" -d Data options: -h, --help show this help message and exit -s, --save save content to data files -dDIRECTORY, --directory=DIRECTORY directory into which to save data files -uURL, --url=URL base URL for requests The command line flags are: s, save Save the content received from each request into a file named with the class and method names for that request. Do *not* perform the unit tests. d, directory *directory* is the directory into which the HTML content files are saved and in which the unit tests look for HTML content files to be used in comparisons. Configuration item: general/directory. Configuration file: section "general", option "directory". u, url *url* is the base URL to which the requests will be made. Note the trailing slash. Configuration item: general/url. Configuration file: section "general", option "url". Configuration options for unit tests ------------------------------------ The unit test program reads a configuration file. The configuration is stored in a file ``unittests.conf`` in the currect directory. Here is an example configuration file for the unit test program:: [general] url=http://localhost:8081/ directory=Data Explanation: - *directory* is the directory into which the HTML content files are saved and in which the unit tests look for HTML content files to be used in comparisons. - *url* is the base URL to which the requests will be made. Note the trailing slash. An example configuration file for unit tests ``unittests.conf`` is included in the distribution. Options specified in the configuration file can be overridden by command line options. See Also ======== `http://www.mems-exchange.org/software/quixote/`_: The Quixote support Web site. .. _`http://www.mems-exchange.org/software/quixote/`: http://www.mems-exchange.org/software/quixote/ `Support for Quixote Etc`_: A variety of helpful documents on Quixote. .. _`Support for Quixote Etc`: http://www.rexx.com/~dkuhlman/quixote_index.html `A Quixote Application Skeleton`_: Example and skeleton files for building a Quixote application. .. _`A Quixote Application Skeleton`: http://www.rexx.com/~dkuhlman/quixote_skeleton.html `Welcome to 'RESTwiki`_: Information and links on REST (REpresentational State Transfer). .. _`Welcome to 'RESTwiki`: http://rest.blueoxen.net/cgi-bin/wiki.pl `Design By Contract`_: Information on the Design by Contract methodology. .. _`Design By Contract`: http://c2.com/cgi/wiki?DesignByContract