Comments on the Development Process

This guide has taken a systematic tour of the design and implementation of the Forum application. The patterns illustrates are generally applicable and can be utilized beyond the world of web-applications, for example in integration solutions, embedded systems, network-edge clients, and categories. To further your understanding of how to conceptualize designing and building applications with NetKernel, review the following general observations on NetKernel development.

URI routing is code: NetKernel is a highly dynamic execution environment. Code is bound together very late; in fact at the last possible moment! NetKernel partitions code in a modular URI address space and locates client code for each request. Because of this flexible and dynamic linking environment, the URI routing, expressed by module interface rules and mapper layers is the skeleton of an application while the individual pieces of code are the organs.

Think local: Applications are more manageable and comprehensible when they are partitioned into local systems. The forum the application is split into natural modular chunks and within each module the application is partitioned into independent channels. Within channels natural layering occurs between atomic services and higher order compositions of those services.

Connect and test: A corollary to the previous two statements is that its good practice to connect code and test the URI routing incrementally, one piece at a time. Minimize the intrusion of bugs by thinking locally and test each one-to-one URI relationship as it is created. These mapping truly are the skeleton of the application, and once code connects locally you can trust that it will stay connected and can move on to consider larger scale relationships.

Use scaffolding: Just as in industrial construction, use and discard scaffolding. Mock a service by using a static resources while you perfect the URI mappings. Once tested, replace the static resource with dynamically linked service calls. The consumer of a URI service does not need to know if it is static or dynamic; to it, it is simply a resource to be used. An example from the Forum is the dynamic link generation - we started with a static file ffcpl:/links.xml and only later did we swap it with a script implementation to generate dynamic links combining the styling resources. Another example is that the web-application can be written against a service module even if the services are not implemented with live data against the database - just have them return a static resource with the right functional form.

Put complexity in black-boxes: As part of thinking local its good practice to push down complexity behind a service interface. It is very cheap to refactor in NetKernel so when a composite assembly of services gets large its probably time to abstract it behind a URI and loosely couple it into the application layers above. We follow a simple mantra: construct, compose, constrain - then there's no room left for complexity. A prominent example from the Forum is how the rendering is treated as a black-box.

Fallback and let the infrastructure catch you: Let the kernel worry about locating resources. You are its master - tell it what to fetch and trust it to deliver. Trust the infrastructure to take care of low-level implementation like concurrency, threading, asynchronicity, caching and compilation.

Emergent complexity versus intelligent design: As Unix has always extolled, simple units can be combined into powerful systems. By working locally and then composing into higher order systems complexity arises naturally, manageable and adaptively without being preconceived and hard-wired. In short you don't need to be able to have a complete top-to-bottom understanding of the entire system provided you understand the detail at the local scale and the broad relationships at the large scale.

Finally, something to consider: as a complete solution I don't know how the Forum works. Surprised? Well something else to consider: the same holds for the designers of the last plane you flew in and the pilots and air-traffic controllers that delivered your travel solution.

Quality of Solution

To conclude lets consider the quality of the forum as a solution. Beyond supporting the required feature set, there are three important time axes to consider when evaluating the quality of a software solution: Time to develop, Time to change and Time to execute. A high quality solution is one which delivers the feature set and for which time is a local minima on all three axes.

You only have my word but the following statements are true:

Time to implement: The forum application was developed in 5 developer days of which 2 days were spent researching the specification, developing the data model and implementing the RDBMS schema. In production for over a year there has been no maintenance overhead and no bug fixes.

Time to change: The forum application was written for production use on the 1060 portal backed by a MySQL database. We decided to include it with the NetKernel Standard Edition distribution as a showcase of some NetKernel design patterns. In order to make it run out of the box we created a new forum-services module to run using the embedded HSQLDB. The forum-services-lite development took less than 1 day and required no modifications to the web application. The web-application can be switched to run on our original MySQL forum-services module or the forum-services-lite module by changing the import definition. In order to look fresher and in-keeping with the NetKernel 3.0 distribution a new style module was developed in 1 hour.

Time to execute: Every XHTML view and resource of the forum is cacheable. Caching occurs naturally at every level of granularity within the application. For example, the dynamically generated XRL links are generated once and cached for all subsequent requests. All scripts and XSL transforms are compiled and cached the first time they are run and subsequently use the highest order form. Cache invalidation is selectively applied using the golden thread pattern without requiring independent channels to be coupled. Due to NetKernel's abstraction caching happens for free it is not a special case and does not have to be coded into the application. The net result: any given piece of code is only executed if the information it would generate does not already exist - by definition then, execution lies in a self optimizing local minima.

Other Factors that increase Quality: There is high overall system robustness due to the underlying infrastructure and the simple uniform abstraction. Finally, NetKernel solutions are coded at a scale that results in much less code which means there are fewer opportunities to write bugs.

Production Use

The pre-installed version of 1060 Forum is deployed with a 'lite' forum-services module which uses the embedded HSQLDB in order that it will run out-of-the-box. HSQL is very convenient and scales reasonably but it is not feature rich (lacks full text search) and there have been reports of stability issues on long living connections and scaling performance cannot be guaranteed.

The Forum is in production-use using MySQL at 1060.org. If you are interested in running your own production Forum you are welcome to install the MySQL-based forum-services using the module installer tool or contact 1060 Research for advice, latest release information and commercial support.

If you're looking for an open-source project and enjoy a challenge - the Forum SQL schema is relatively portable and should be easy to port to other well-known RDBMS. As mentioned above, the MySQL and HSQL-based forum-services modules are plug replaceable with the front-end web-application so no work is needed to modify the forum-web module. If you do develop a port, please let us know and we'll make it available for others to use.