Dave Kuhlman
http://www.rexx.com/~dkuhlman
Email: [email protected]
AOLserver, PyWX, and PostgreSQL provides a powerful platform for building Web applications. This document is a teaching document that intended to help the user to build, install, configure, and use each of these three tools.
These notes describe my experiences with building, installing, configuring, and running AOLserver, PyWX, and PostgreSQL. Needless to say, ``Your mileage may vary''. My system is GNU/Linux from Libranet, which is based on the Debian Linux distribution.
Obtain the AOLserver source code from:http://aolserver.com/.
Expand the source code with something like the following (depending on the version that you obtain):
tar tzf aolserver-3.5.0-src.tar.gz
To configure, build, and install the AOLserver software, use something like the following:
./configure --prefix=/usr/local/aolserver \ --with-tcl=/usr/local/lib \ --enable-threads make make install # as root
Comments:
Configuration of AOLserver is done in the file A sample configuration file nsd.tcl is placed in the root directory of the directory tree where you installed AOLserver. For example, the above configuration and build instructions would install AOLserver in /usr/local/aolserver and the sample configuration file in /usr/local/aolserver/nsd.tcl.
Configuration is done in a Tcl file. In modifying nsd.tcl you will need to follow Tcl syntax and rules. If you do not know Tcl or need a refresher, you can look here: http://www.scriptics.org. The Tcl book is also a good source: Tcl and the Tk Toolkit, by John K Ousterhout.
Notes on changes to configuration file nsd.tcl:
set httpport 8081
set hostname [ns_info hostname]
to:
set hostname "warbler"
or something like it.
ns_section "ns/server/${servername}/module/nscgi" set cgidir "${homedir}/cgi" ns_param map "GET /cgi ${cgidir}" ;# CGI script file dir (GET). ns_param map "POST /cgi ${cgidir}" ;# CGI script file dir (POST).
This specifies, for example on my machine, that a request of the form "http://warbler:8081/cgi/test1.py" would be resolved to ./aolserver/cgi/test1.py. (``warbler'' is the hostname of my system.)
Note that we will be making more modifications to the AOLserver configuration file when we describe the configuration of PyWX and PostgreSQL.
Here is what I use to start AOLserver on my machine:
/usr/local/aolserver/bin/nsd -ft /usr/local/aolserver/nsd.tcl
Notes and suggestions:
chown
).
PyWX enables you to use Python as your scripting language for the implementation of dynamically generated Web pages. AOLserver's primary support is for the use of Tcl and C. That is, the APIs exposed by AOLserver are accessible by Tcl and C. PyWX enables you to use Python as a first class scripting language on top of AOLserver.
Note: Also see the file src/INSTALL for more information on how to build and install PyWX.
And here are steps that you can follow:
make distclean ./configure --with-threads make OPT="-fpic -O2" mkdir .extract (cd .extract; ar xv ../libpython2.2.a) gcc -shared -o libpython2.2.so .extract/*.o rm -rf .extract
Comments:
./configure --with-python=2.2 --with-aolserver-src=/Source/aolserver-3.5.1-src
Notes:
#PYTHONDIR = $(includedir)/@PYTHONDIR@ PYTHONDIR = $(includedir)/python2.2 #AOLSVRDIR = $(includedir)/@AOLSVRDIR@ AOLSVRDIR = /usr/local/aolserver/include #AOLSERVER_VERSION = -DAOLSERVER_VERSION_3_2 AOLSERVER_VERSION = -DAOLSERVER_VERSION_3_5 #LIBS = -lpthread -ldl LIBS = -L/usr/local/aolserver/lib -lpthread -ldl -lpython2.2 -lutil -lstdc++
The ``-lpython2.2'' links in the Python shared library that we built earlier. And, ``-L/usr/local/aolserver/lib'' specifies where to find that library.
make
Below is a script that copies the PyWX files to the correct locations on my machine. You might consider taking this one and editing it before using it. For example, change ``server1'' to the name of your server in both of the following scripts.
Also, before running the script, you will need to make the pywx-lib directory under the aolserver directory. Do something like the following:
mkdir /usr/local/aolserver/pywx-lib
Then run the following (after suitable editing, of course) each time you build PyWX to copy the newly built files to your installation location:
#!/bin/sh -x PyWXPath=/usr/local/aolserver/pywx-lib cp src/nspywx.so /usr/local/aolserver/bin chmod 755 /usr/local/aolserver/bin/nspywx.so cp pywx-lib/*.py ${PyWXPath} cp src/Nscmodule.so ${PyWXPath}
Here is another version of the above script that (1) creates the needed directory automatically and (2) also changes the owner of the installed files. Use this one if you want to run AOLserver under a different user.
#!/bin/sh -x PyWXPath=/usr/local/aolserver/pywx-lib rm /usr/local/aolserver/bin/nspywx.so cp nspywx.so /usr/local/aolserver/bin mkdir ${PyWXPath} cp pywx-lib/*.py ${PyWXPath} cp src/Nscmodule.so ${PyWXPath} chown nsadmin ${PyWXPath}/* chgrp nsadmin ${PyWXPath}/* chwon nsadmin /usr/local/aolserver/bin/nspywx.so chgrp nsadmin /usr/local/aolserver/bin/nspywx.so chmod 755 /usr/local/aolserver/bin/nspywx.so
Most of the configuration for PyWX is done in the AOLserver configuration file. In my case, this is done in /usr/local/aolserver/nsd.tcl.
The PyWX INSTALL file contains instructions on what to add to the AOLserver configuration file. The file config/pywx-examples.tcl contains a template for the Tcl code to be added to the AOLserver configuration file. You can either copy the contents of config/pywx-examples.tcl into your AOLserver configuration file, or you can copy config/pywx-examples.tcl to your AOLserver install directory and include it into your AOLserver configuration file with something like the following:
source "${homedir}/pywx-config.tcl"
What follows are notes on modifications to the additions you will need to make to your copy of file config/pywx-examples.tcl.
ns_section "ns/server/${servername}/module/pywx"
put something like:
ns_param PyWXPath "${homedir}/pywx-lib"
ns_param Map "* /examples/test/*.py pywx.runscript"
What this says is: for all HTTP methods (``*''), if a .py file is in directory examples/test under my ``pageroot'' (/usr/local/aolserver/servers/server1/pages on my system) directory, then run the file pywx.runscript to start it up. Effectively, you are telling AOLserver that whenever a .py file is requested from directory pages/examples/test, AOLserver should deliver it by running it, not by delivering the contents of that .py file.
This section describes the process creating and testing a small CGI script written in Python using PyWX. Here are steps that you can follow:
#!/usr/bin/env python import os def test(): variables = os.environ['QUERY_STRING'] print 'Content-type: text/html' print '' print '<html>' print '<head>' print '<title>AOLserver CGI test #1</title>' print '</head>' print '<body>' print '<p>AOLserver CGI test #1</p>' print '<p>Hello, world</p>' print '<p>CGI variables: %s</p>' % variables print '</body>' print '</html>' print '' test()
ns_section "ns/server/${servername}/module/nscgi" set cgidir "${homedir}/cgi" ns_param map "GET /cgi ${cgidir}" ;# CGI script file dir (GET). ns_param map "POST /cgi ${cgidir}" ;# CGI script file dir (POST).
If the value of ${homedir} is /usr/local/aolserver, for example, as it is on my system, then we would place CGI scripts in /usr/local/aolserver/cgi.
chmod a+x testcgi1.py
Admittedly, this is a trivial script. It's only purpose to to determine whether we have installed and configured AOLserver and PyWX correctly for CGI.
More CGI samples and suggestions on how to accomplish specific tasks are in the ``Hints and Code Samples'' section below.
This section describes how to write and run a small script that dynamically generates a Web page using Python and PyWX. You can follow these steps:
#!/usr/bin/env python import string import Ns def test(): conn = Ns.GetConn() lines = [] lines.append('<html>') lines.append('<head>') lines.append('<title>AOLserver non-CGI test #1</title>') lines.append('</head>') lines.append('<body>') lines.append('<p>AOLserver <b>non-CGI</b> test #1</p>') lines.append('<p>Hello, world</p>') lines.append('<p>CGI variables:</p>') try: query = conn.GetQuery() except: query = None lines.append('<ol>\n') if query: size = query.Size() for idx in range(size): key = query.Key(idx) value = query.Value(idx) lines.append('<li>Key: "%s" Value: "%s"</li>\n' % (key, value)) lines.append('</ol>\n') lines.append('</body>') lines.append('</html>') lines.append('') content = string.join(lines, '\n') conn.ReturnHtml(200, content) test()
A few comments and a little explanation on the above script:
for
statement that iterates over this set.
try:except
.
set pageroot ${homedir}/servers/${servername}/pages
On my system, where AOLserver is installed in /usr/local/aolserver and my server name is ``server1'', the pageroot is /usr/local/aolserver/servers/server1/pages. I've made examples/test subdirectories under my page root. So, in my testing, I placed testnoncgi.py in /usr/local/aolserver/servers/server1/pages/examples/test/testnoncgi.py.
chmod a+x testnoncgi1.py
Here is what I did:
tar xvzf postgresql-driver.tgz
You can also find guidance for installing PostgreSQL and configuring it for AOLserver at http://pascal.scheffers.net/openacs/pgupdate/index.html.
Here is a short script to test access to your PostgreSQL database.
#!/usr/bin/env python import string import Ns def test(): doc = [ '<html>', '<head><title>testpostgres.py</title></head>', ] conn = Ns.GetConn() server = conn.Server() pool = Ns.DbPoolDefault(server) dbHandle = Ns.DbHandle(pool) # Change "plants" in next line to the name of a table in your DB. set = dbHandle.Select('select * from plants order by name') doc = [] doc.append('<html>') doc.append('<head><title>Testing PostgreSQL</title></head>') doc.append('<body>') doc.append('<h1>Testing PostgreSQL</h1>') doc.append('<p>Plants Table</p>') doc.append('<table border="a">') doc.append('<tr>') doc.append('<th rowspan="2">#</th>') for i in range(len(set)): doc.append('<th>%d</th>' % (i,)) doc.append('</tr>') doc.append('<tr>') for k in set.keys(): doc.append('<th>%s</th>' % (Ns.QuoteHtml(k),)) doc.append('</tr>') i = 0 while dbHandle.GetRow(set) == Ns.OK: doc.append('<tr>') doc.append('<th>%d</th>' % (i,)) values = set.values() doc.append('<td>%s:</td><td>"%s"</td>' % \ (values[0], values[1])) doc.append('</tr>') i += 1 if i > 100: break doc.append('</table>') doc.append('</body>') doc.append('</html>') content = string.join(doc, '\n') conn.ReturnHtml(200, content) test()
The above script displays the contents of the table ``plants'' in my database.
Comments:
content = string.join(doc, '\n')
conn.ReturnHtml(200, content)
for k in set.keys(): doc.append('<th>%s</th>' % (Ns.QuoteHtml(k),))
while dbHandle.GetRow(set) == Ns.OK: doc.append('<tr>') doc.append('<th>%d</th>' % (i,)) values = set.values() doc.append('<td>%s:</td><td>"%s"</td>' % \ (values[0], values[1])) doc.append('</tr>')
iterates over the selected rows, gets the values (fields) in each row, and generates a line of content for each row.
The following sample code checks for the existence of a row whose field ``name'' has a given value.
def testCheck(name): name = None doc = [] conn = Ns.GetConn() try: query = conn.GetQuery() except: query = None if query: name = query.Get('name') doc.append('<html>') doc.append('<head><title>Testing PostgreSQL</title></head>') doc.append('<body>') doc.append('<h1>Testing PostgreSQL</h1>') if not name: doc.append('<p>No name provided</p>') else: server = conn.Server() pool = Ns.DbPoolDefault(server) dbHandle = Ns.DbHandle(pool) try: row = dbHandle.Db1Row("select * from plants where name = '%s'" \ % name) except: row = None if row: value = row.Get('descrip') doc.append( '<p>Name "%s" with value "%s" exists in table "plants".</p>' % \ (name, value)) else: doc.append('<p>Name "%s" does not exist in table "plants"</p>.' % \ name) doc.append('</body>') doc.append('</html>') content = string.join(doc, '\n') conn.ReturnHtml(200, content) testCheck(name="orange")
Comments:
Db0or1Row
instead of Db1Row
.
However, it is more complicated than Db1Row
and it did not
work. I was not able to get the values in the row returned.
content = string.join(doc, '\n') conn.ReturnHtml(200, content)
converts the list (of content) to a single string, and returns it to be send to the client.
This example demonstrates how to check for and then delete a row from a table in the database.
def testdelete(name): name = None doc = [] conn = Ns.GetConn() try: query = conn.GetQuery() except: query = None if query: name = query.Get('name') doc.append('<html>') doc.append('<head><title>Testing PostgreSQL</title></head>') doc.append('<body>') doc.append('<h1>Testing PostgreSQL</h1>') if not name: doc.append('<p>No name provided</p>') else: server = conn.Server() pool = Ns.DbPoolDefault(server) dbHandle = Ns.DbHandle(pool) try: row = dbHandle.Db1Row("select * from plants where name = '%s'" \ % name) except: row = None if row: value = row.Get('descrip') dbHandle.Exec("delete from plants where name='%s'" % name) doc.append('<p>Name "%s" deleted from table "plants".</p>' % \ name) else: doc.append('<p>Name "%s" does not exist in table "plants".</p>' % \ name) doc.append('</body>') doc.append('</html>') content = string.join(doc, '\n') conn.ReturnHtml(200, content) testdelete(name="orange")
Comments:
dbHandle.Exec("delete from plants where name='%s'" % name)
This example demonstrates how to check for and then update an existing row in a table in the database.
def testupdate(name, descrip): conn = Ns.GetConn() doc = [] app = doc.append doc.append('<html>') doc.append('<head><title>Testing PostgreSQL</title></head>') doc.append('<body>') doc.append('<h1>Testing PostgreSQL</h1>') name = None try: query = conn.GetQuery() except: query = None if query: value = query.Get('name') if value: name = value value = query.Get('descrip') app('<p>value: "%s"</p>' % value) if value: descrip = value if not name: doc.append('<p>No name provided</p>') else: server = conn.Server() pool = Ns.DbPoolDefault(server) dbHandle = Ns.DbHandle(pool) try: row = dbHandle.Db1Row("select * from plants where name = '%s'" \ % name) except: row = None if row: dbHandle.Exec("update plants set descrip='%s' where name='%s'" % \ (descrip, name)) doc.append('<p>Name "%s" updated in table "plants".</p>' % \ name) doc.append('<p>New descrip "%s".</p>' % descrip) else: doc.append('<p>Name "%s" does not exist in table "plants".</p>' % \ name) doc.append('</body>') doc.append('</html>') content = string.join(doc, '\n') conn.ReturnHtml(200, content) testupdate(name="orange", descrip="good food")
Comments:
dbHandle.Exec("update plants set descrip='%s' where name='%s'" % \ (value, name))
This is a slightly more extensive example that enables the end user to enter new rows, to update existing rows, and to delete existing rows.
This script follows these rules:
#!/usr/bin/env python # # testpostgres.py # import sys, string, time import Ns DEBUG = 1 def logmsg(msg): if DEBUG: # Change the next line to point to a directory that we can # write to. logfile = file('/w2/Tmp/Logs/test.log', 'a') logfile.write('[%s] testpostgres.%s\n' % (time.ctime(), msg)) logfile.close() def updateDB(queryDict, dbHdl): ## logmsg('updateDB queryDict: ' + str(queryDict)) name = None descrip = None if queryDict.has_key('plant_name'): name = queryDict['plant_name'] if queryDict.has_key('plant_descrip'): descrip = queryDict['plant_descrip'] if name and descrip: # We have both name and description. # Update a row if it exists, else insert a new row. rowSet = dbHdl.Select("select * from plants where name = '%s'" % name) if dbHdl.GetRow(rowSet) == Ns.OK: s1 = "update plants set descrip='%s' where name='%s'" % \ (descrip, name) dbHdl.Exec(s1) else: s1 = "insert into plants values('%s', '%s')" % (name, descrip) dbHdl.Exec(s1) elif name: # We have only the name (but not the description). # Delete the DB entry for name. rowSet = dbHdl.Select("select * from plants where name = '%s'" % name) if rowSet.Size() > 0: s1 = "delete from plants where name='%s'" % name dbHdl.Exec(s1) else: # Do nothing because neither name nor descrip were entered. pass def test(): conn = Ns.GetConn() doc = [ '<html>', '<head><title>testpostgres.py</title></head>', '<body>', '<form method="POST" action="testpostgres.py">', '<div align="center"><h1>testpostgres.py</h1></div>', ] server = conn.Server() pool = Ns.DbPoolDefault(server) dbHandle = Ns.DbHandle(pool) try: query = conn.GetQuery() except: query = None queryDict = {} if query: variables = query size = variables.Size() for idx in range(size): key = variables.Key(idx) value = variables.Value(idx) queryDict[key] = value updateDB(queryDict, dbHandle) set = dbHandle.Select('select * from plants order by name') doc.append('<table border="1" cellpadding="1">') doc.append('<tr>') doc.append('<th rowspan="2">#</th>') for idx in range(len(set)): doc.append('<th>%d</th>' % (idx,)) doc.append('</tr>') doc.append('<tr>') for k in set.keys(): doc.append('<th>%s</th>' % (Ns.QuoteHtml(k),)) doc.append('</tr>') idx = 0 while dbHandle.GetRow(set) == Ns.OK: doc.append('<tr>') doc.append('<th>%d</th>' % (idx,)) values = set.values() doc.append('<td>%s:</td><td>"%s"</td>' % \ (values[0], values[1])) doc.append('</tr>') idx += 1 if idx > 100: break doc.append('</table>') doc.append('<hr>') doc.append('<table>') doc.append('<tr>') doc.append('<td>') doc.append('Name:') doc.append('</td>') doc.append('<td>') doc.append('<input type="TEXT" name="plant_name" value="">') doc.append('</td>') doc.append('</tr>') doc.append('<tr>') doc.append('<td>') doc.append('Descrip:') doc.append('</td>') doc.append('<td><input type="TEXT" name="plant_descrip" value=""></td>') doc.append('</tr>') doc.append('<td>') doc.append('<input type="SUBMIT" value="Submit form">') doc.append('</td>') doc.append('</tr>') doc.append('</table>') doc.append('</form>') doc.append('</body>') doc.append('</html>') content = string.join(doc, '\n') conn.ReturnHtml(200, content) test()
Comments:
This section explains how to use Quixote templates and the Quixote template language PTL (Python template language) on top of PyWX. This brings PyWX one step closer to a Web framework.
You should be aware that there is an adapter for Quixote that enables you to use all of the capabilities of Quixote with PyWX, whereas this section explains how to use a single Quixote feature, specifically templates. One of the points of this section is to show that PyWX is extensible and flexible. You, in particular, may choose to use Cheetah or some other Python template language instead of PTL.
Here is how you can do it:
First, install Quixote. You can find Quixote at http://www.mems-exchange.org/software/quixote/.
Then, for each page that you want to use PTL, do the following:
from quixote import enable_ptl
enable_ptl()
import mytemplate o o o mytemplate.template1()
Comments:
The following is a more complete, though simple, example. Here is the (plain) Python file (testtemplate.py):
#!/usr/bin/env python import sys import Ns from quixote import enable_ptl enable_ptl() curPath = '/usr/local/aolserver/servers/server1/pages/examples/test1' if not curPath in sys.path: sys.path.append(curPath) import testtemplate1 def test(): conn = Ns.GetConn() content = testtemplate1.testtemplate(3) conn.ReturnHtml(200, content) test()
And here is the sample PTL file (testtemplate1.ptl):
import sys import Ns template testtemplate(maxLines): """\ <html> <head> <title>Test PTL on AOLserver/PyWX</title> </head> <body> <div align="center"><h1>Test PTL on AOLserver/PyWX</h1></div> <p>sys.path:</p> <ul> """ count = 0 for path in sys.path: count += 1 if count > maxLines: break '<li>%s</li>\n' % path """\ </ul> </body> </html> """
Comments:
enable_ptl()
installs the Quixote ``import hook'' that
enables us to import PTL files.
sys.path
before importing the template. The
if
statement protects us from adding it yet again each time
the page is visited.
This section describes how to use the Quixote Web app server on top of PyWX and AOLserver.
Our goal here is to make it possible to use a variety of application delivery models on top of AOLserver and PyWX. A Web app server such as Quixote supports at least one of those models.
Here is how to do it:
There is a handler in the PyWX distribution (pywx-lib/handleQuixote.py). Here is a module handleQuixote containing a sample handler class Handler:
# # handleQuixote.py # import sys import quixote quixote.enable_ptl() import os import ns_setup import PyWX_buffer class Handler: """ Base handler for Quixote applications inside of PyWX. """ def __init__(self, base_url, app): if base_url[0] != '/': raise Exception("base_url must be an absolute local URL.") self._base_url = base_url sys.argv = (base_url,) assert isinstance(app, quixote.Publisher) self._app = app def handle(self, conn, add_environ = {}): # Set up a CGI-style environment: environ = {} environ.update(os.environ) ns_setup.create_cgi_environ(conn, environ) # Set a few parameters that Quixote expects to be munged in a # particular way: environ['SCRIPT_NAME'] = self._base_url environ['PATH_INFO'] = conn.request.url[len(self._base_url):] # Get an I/O handle for this connection: buff = PyWX_buffer.GetBuff() try: # publish! self._app.publish(buff, buff, buff, environ) buff.close() except IOError: # be forgiving of closed connections. pass
Comments:
Here is a sample handler:
#!/usr/bin/env python # # handler.py # from quixote import enable_ptl, Publisher import handleQuixote appPath = "/usr/local/aolserver/servers/server1/pages/Qtest/Test1" baseUrl = '/Qtest/Test1' configFile = '%s/qtest1.conf' % appPath app = Publisher(appPath) app.read_config(configFile) app.setup_logs() handler = handleQuixote.Handler(baseUrl, app) def handle(conn): handler.handle(conn)
Here are a few comments on what handler.py does:
enable_ptl()
,
we will be able to use and import Quixote .ptl
files.
ns_section "ns/server/${servername}/module/pywx"
ensure that you have:
ns_param Pools "interp,thread,cgi"
ns_param PyWXPath "${homedir}/pywx-lib"
So, I put the handleQuixote module and the handler.py handler in the directory /usr/local/aolserver/pywx-lib.
ns_section "ns/module/pywx/pool/thread"
put the following:
ns_param Mode Threaded ns_param PythonPath "pathToQuixote:pathToYourAppCode" ns_param Setup "import handler"
where:
The "ns_param Setup "import handler"" line imports your handler from handler.py.
ns_section "ns/server/${servername}/module/pywx/pool/thread"
place something like the following:
ns_param Map "* /Qtest/Test1/* handler.handle"
What does this say? It tells AOLserver to call
handler.handle()
whenever it receives a request whose base
URL is /Qtest/Test1. Further more, because of the base URL
used in handler.py, the call to handler.handle()
will
result in code being executed from Qtest/Test1 under my
$homedir/servers/$servername/pages directory.
Note that if you are exposing more than one Quixote application, you will have multiple "ns_param Map" lines in your configuration file and you may have to add another handler.py (with another name, of course) as well. However, in the current implementation, only one can be active at a time; comment out the others.
ns_section "ns/threads"
add the following:
ns_param stacksize [expr 128*1024]
In the sample-config.tcl that I used as the basis for my nsd.tcl, this line was commented out.
And the rest is a matter of implementing a Quixote application. But, I'll let you go to the Quixote documentation for that.
Titus Brown, Brent Fulgham, Michael Haggerty -- The developers of PyWX. Titus is currently actively supporting PyWX and makes this possible.
John Totten -- John did a heavy review of this document in the process of using it. He made many very good suggestions and helped improve this document by pointing out problems, errors, and omissions..
See Also:
This document was generated using the LaTeX2HTML translator.
LaTeX2HTML is Copyright © 1993, 1994, 1995, 1996, 1997, Nikos Drakos, Computer Based Learning Unit, University of Leeds, and Copyright © 1997, 1998, Ross Moore, Mathematics Department, Macquarie University, Sydney.
The application of LaTeX2HTML to the Python documentation has been heavily tailored by Fred L. Drake, Jr. Original navigation icons were contributed by Christopher Petrilli.