Chapter 43. The Persistent Connection

Table of Contents

1. Overview
2. Architectural Limitations on Scalability
3. The connection manager
3.1. attributes
3.2. methods
4. The connection datasource
4.1. LzConnectionDatasource methods
4.2. events
5. Agents
5.1. Receiving data
5.2. Sending data
5.3. Getting a list of connected users
6. Authentication
6.1. AuthenticationServlet
6.2. Skipping authentication
6.3. Implementing an authenticator
7. Appendix
7.1. Glossary
7.2. Demo applications
[Note] Note

This feature is provisional. It works in limited capacity situations and is fine to develop with, but it is not recommend deployment (with the possible exception of low-capacity, non-mission critical deployment) with this feature.

[Warning]
[SWF]

The Persistent Connection Manager works in the Flash runtime only. It is not supported in DHTML.

This document describes persistent connection concepts, definitions, and elements. It also demonstrates how to program in LZX using the persistent connection. It assumes you have a basic knowledge of LZX (in particular, datasets and datapointers) and a basic understanding of servlet containers and HTTP cookies/sessions.

1. Overview

A persistent connection is a one-way pipe held by an application from the OpenLaszlo Server. Because all transactions are done through an HTTP port, it allows an application to receive asynchronous messages from the server without the need to open up a port in the firewall. Maintanance of an application's persistent connection is done by the connection manager, whose job is to multiplex all incoming data into a connection dataset. A connection datasource is used to declare connection datasets. Each connection datasource registers itself with the connection manager. This allows the connection manager to look through its list of connection datasources to see if there's a dataset that matches the destination of an incoming message. It's possible that two separate datasets receive the same message if different datasources declare datasets with the same name. Assuming the following:

<connectiondatasource name="one">
  <dataset name="message">
  <dataset name="alert">
</connectiondatasource>

<connectiondatasource name="two">
  <dataset name="stock">
  <dataset name="alert">
</connectiondatasource>

datasets one.alert and two.alert will receive the same data for messages destined for "alert".

2. Architectural Limitations on Scalability

The persistent connection does hold a socket open; but it's an HTTP connection. This makes it work with firewalls, etc —often, non-port 80 traffic will be blocked. This technique is sometimes called "HTTP trickle" and it's used in systems other than OpenLaszlo, such as KnowNow, mod_pubsub, Kenemea, etc. A single HTTP request is made to the OpenLaszlo server; that connection remains open and the server trickles info back (1) when a real-time event is being pushed to the client, and (2) just to keep the connection alive (a minimal amount of info is returned every few seconds in the absence of pushed events).

However, the persistent connection feature is incomplete. HTTP connections have their own issues (such as incompatibility with Safari, and excess consumption of server resources, at least as currently implemented). That hasn't stopped OpenLaszlo users from deploying applications that use this feature, but it isn't adequate for a high-scale deployment.

Another approach would be to "drop down" into Flash and use the Flash socket API as its basis. The persistent connection might first try the socket, and if that failed, fall back to HTTP trickle (at least when the OpenLaszlo server is part of the deployment); such an approach is not part of Openlaszlo now, but applications using that architecture have been developed and deployed. We mention it here merely as a hint to developers who might want to extend the persistent connection manager on their own.

3. The connection manager

The connection manager is an LzConnection object that is instantiated when <connection> is declared in the canvas (note: <connection> can't be declared in a library). Once constructed, the connection manager can be accessed using canvas.connection.

Example 43.1. Simple connection manager

  <canvas height="120" debug="true">
      <connection/>
      <script>
          Debug.write('my connection manager: ' + canvas.connection);
      </script>
  </canvas>

To establish a persistent connection to the OpenLaszlo Server, use the LzConnection.connect() method.

  <canvas>
      <connection authenticator="anonymous"/>
      <script>
        canvas.connection.connect();
      </script>
  </canvas>

An anonymous authenticator tells the connection that it doesn't require authentication. An authenticator ensures connection requests to the server are authorized. By default, the connection validates its request is valid by calling a back-end authentication server. See Authentication for more details.

Once a connection is established, it is identified with a username and a unique connection session id. The username gives the connection a target for other clients to use for sending messages. Multiple connection sessions may have the same username. You can get these values by calling LzConnection.getUsername() and LzConnection.getSID().

3.1. attributes

You can modify the behavior of the persistent connection by changing attributes in the connection tag. The available attributes are:

group

used to name the connection group an application's persistent connection belongs to

securesecureport

used to establish a secure persistent connection

heartbeat/timeout

sets the frequency at which the OpenLaszlo Server checks to see if an application is still listening to the persistent connection

receiveuserdisconnect

if true, receives notification from the OpenLaszlo Server whenever a client closes their persistent connection

authenticator/authparam

the java authentication class to use in the OpenLaszlo Server for connection requests

3.1.1. group

A connection group is a set of connected applications. Messages sent to a group can be received by other connections registered to that group. A connection with an unspecified group name has its webapp path as its group. You can use the group attribute to register your connection to a group.

<connection group="mygroup"/>

Clients not registered the same group are not allowed to send messages to each other.

3.1.2. secure/secureport

To establish a secure persistent connection, set the secure attribute to true. Your server must be configured to allow HTTPS requests. You can also specify a different port if your SSL listener is not on the standard 443 port.

<connection secure="true" secureport="8443"/>

Setting the secure attribute also secures connection functions that make requests to the OpenLaszlo Server, for example, LzConnection.connect(), LzConnectionDatasource.sendMesage(), etc.

3.1.3. heartbeat/timeout

By default, the OpenLaszlo Server verifies a connection is alive by sending a heartbeat every 5 seconds. The connection is configured to timeout if a message or heartbeat hasn't been heard from the server in 20 seconds. You can change these values using the heartbeat and timeout attributes, whose values are in milliseconds.

<connection heartbeat="10000" timeout="30000"/>

3.1.4. receiveuserdisconnect

An application can be notified through the persistent connection whenever another client disconnects. To turn this feature on, just set receiveuserdisconnect to true.

<connection receiveuserdisconnect="true"/>

This feature is false by default.

3.1.5. authenticator/authparam

An authenticator is a server-side object that validates connection requests. By default, the connection uses org.openlaszlo.auth.HTTPAuthentication. By default, this authenticator contacts the provided authentication servlet located in http://.../WEBAPP/AuthenticationServlet, where WEBAPP is the web application path where you installed the OpenLaszlo Server (typically, lps-4.0.3).

You can use the authenticator attribute to use a different authenticator for your connection. This attribute expects a java class name. When the first connection request is made, the server instantiates an object based on the java class specified. Subsequent calls uses this java authentication object to authenticate requests. Parameters can be passed into the java authenticator using the authparam attribute, which expects a query-styled string. To skip authentication altogether, you can set authenticator="anonymous". All connections are named "user" unless you set authparam="myusername".

See Authentication for more details.

3.2. methods

3.2.1. login(name, password)

Login allows you to session your application. The connection manager uses its authenticator for sessioning. Cookies and response headers returned by an authentication server will be set in the application. The return value of this call is returned in connection.loginDset with a root element of <login> like:

<login status="message">
  loginXML
</login>

See HTTPAuthentication Login for the login XML return.

3.2.2. The logout() method

Use the logout() method to remove an application's session. The return value of this call is returned in connection.logoutDset with a root element of <logout like:

<logout status="message">
  logoutXML
</logout>

See HTTPAuthentication Logout for the logout XML return.

3.2.2.1. Example:

This demonstrates how to session and unsession your application.

<canvas debug="true" height ="200"  >
  <debug y="60"/>
  <connection/>

  <datapointer xpath="connection:loginDset:/login[1]/authentication[1]/response[1]/status[1]">
    <handler name="ondata">
      var statusMessage = this.xpathQuery('@msg');
      Debug.write('login: ' + statusMessage);
    </handler>
    <handler name="onerror">
      Debug.write('login resulted in error');
    </handler>
    <handler name="ontimeout">
      Debug.write('login timed out');
    </handler>
  </datapointer>

  <datapointer xpath="connection:logoutDset:/logout[1]/authentication[1]/response[1]/status[1]">
    <handler name="ondata">
      var statusMessage = this.xpathQuery('@msg');
      Debug.write('logout: ' + statusMessage);
    </handler>
    <handler name="onerror">
      Debug.write('logout resulted in error');
    </handler>
    <handler name="ontimeout">
      Debug.write('logout timed out');
    </handler>
  </datapointer>

  <view>
    <simplelayout axis="x" spacing="5"/>
    <button>login
      <handler name="onclick">
        canvas.connection.login('adam', 'adam');
      </handler>
    </button>
    <button>logout
      <handler name="onclick">
        canvas.connection.logout();
      </handler>
    </button>
  </view>
</canvas>

3.2.3. The connect() method

To establish a connection, you can use the connect() method. The first time connect() is called, the connection manager establishes the connection. Since only once persistent connection can exist per application, subsequent calls to connect() will close the previous connection and establish a new one.

3.2.4. The disconnect() method

Use disconnect() to close the persistent connection. This function calls the server to immediately inform it to close down the connection, as well as closing down the client-side connection. If you just want to close down the client-side connection, use clientDisconnect(). The benefit to using disconnect() is that the server is immediately informed of the disconnection instead of having to wait for the next heartbeat. You can view the return status of disconnect() using connection.disconnectDset dataset.

3.2.4.1. Example:

This demonstrates how to use connect(), disconnect(), and clientDisconnect().

Example 43.2. Different ways to disconnect

<canvas debug="true" height="200">
  <debug y="60"/>
  <connection authenticator="anonymous">
    <handler name="onconnect">
      Debug.write('connected');
    </handler>
    <handler name="ondisconnect">
      Debug.write('client disconnected');
    </handler>
  </connection>

  <datapointer xpath="connection:disconnectDset:/*[1]">
    <handler name="ondata">
      Debug.write('server disconnected');
    </handler>
    <handler name="onerror">
      Debug.write('disconnect call to server resulted in error');
    </handler>
    <handler name="ontimeout">
      Debug.write('disconnect call to server timed out');
    </handler>
  </datapointer>

  <view>
    <simplelayout axis="x" spacing="5"/>
    <button>connect
      <handler name="onclick">
        canvas.connection.connect();
      </handler>
    </button>
    <button>disconnect
      <handler name="onclick">
        canvas.connection.disconnect();
      </handler>
    </button>
    <button>client disconnect
      <handler name="onclick">
        canvas.connection.clientDisconnect();
      </handler>
    </button>
  </view>
</canvas>

4. The connection datasource

While the connection tag configures the behavior of the persistent connection, the connection datasource is where you declare the datasets that handle incoming data. A connection datasource is a LzConnectionDatasource object. Unlike the connection tag, connection datasources and datasets can be declared in libraries.

<library>
  <connectiondatasource name="mydatasource">
    <dataset name="message"/>
    <dataset name="alert"/>
  </connectiondatasource>
</library>

When a connection datasource is declared, it registers itself with the connection manager. In turn, The connection manager routes messages received from the connection to a connection datasource's dataset.

Since connection datasets only receive data, they will only raise ondata events. You can handle events raised by the persistent connection through the connection manager or a connection datasource.

4.1. LzConnectionDatasource methods

Many methods in LzConnectionDatasource return their results in a dataset.

Table 43.1. LzConnectionDatasource Methods

Methods Result dataset
sendMessage(toList, mesg, dest) sendMessageDset
sendXML(toList, xml, dest) sendXMLDset
sendUserXML(userList, xml, dest) sendXMLDset
sendAgentXML(agentList, xml) sendXMLDset
getList(userList) getListDset

You can listen for a response status by declaring a datapointer to the corresponding dataset. An example is shown below.

Example 43.3. Sending message over connection

<canvas debug="true" height="200">
  <debug y="60"/>
  <connection authenticator="anonymous"/>


  <script>
      connection.connect();
  </script>

  <connectiondatasource name="myconnection">
      <dataset name="messages"/>
  </connectiondatasource>

  <datapointer xpath="myconnection:messages:/*[1]">
    <handler name="ondata">
      var from = this.xpathQuery('/from[0]/@name');
      var mesg = this.xpathQuery('/text()');
      Debug.write('got data from ' + from +  ', mesg is ['+ mesg + ']!');
    </handler>
  </datapointer>
  
  <datapointer xpath="myconnection:sendMessageDset:/*[1]">
    <handler name="ondata">
      Debug.write("*** got result back ***");
    </handler>
  </datapointer>
  
  <button>send message
    <handler name="onclick">
      myconnection.sendMessage('*', 'a message', 'messages');
    </handler>
  </button>
</canvas>

4.1.1. sendMessage() / sendXML()

There are two ways to send data to other clients: sendMessage() and sendXML(). Each send function takes three parameters:

  • comma-separated list of destination clients

  • message

  • destination dataset where the message should be pushed into

The sendMessage() method sends a message string. The message is received by other clients like:

<root dset="aDataset">
   <from name="sender"/>
   message string
</root>

The sendXML() call sends arbitray XML data. The sent data looks like:

<root dset="aDataset">
  <XML>
</root>

The results of the call are returned in the connection datasource's sendMessageDset and sendXMLDset, respectively. Both sendMessage()() and sendXML()() send data to users and agents. You can narrow that domain by using sendUserXML() and sendAgentXML(). Users are peer applications and agents are back-end servers.

The XML result from a sent request looks like:

<send count="sent-messages"/>

If there's a response from any agent, the resultset can look like:

<send count="sent-messages">
  <agent name="agent1">
    Agent1XML
  </agent>
  <agent name="agent2">
    Agent2XML
  </agent>
  ...
</send>

See "Agents" for details on connection agents.

4.1.2. getList()

To get a list of connected usernames, use the getList() call. The function takes a comma-separated list of usernames, or use * for all users. The result is returned in the getListDset dataset like:

<list>
  <user name="name1">
  <user name="name2">
  <user name="name3">
  ...
</list>

4.2. events

The connection manager and connection datasource can handle these connection events:

onconnect

raised as soon as an established connection is verified.

ondata

raised whenever new data arrives for one of the datasource's datasets. The root node of the data is returned along with the ondata event and looks like:

<root dset="dataset">
  arbitraryXML
</root> 

onerror

raised if there was a problem establishing the persistent connection. An error XML is returned with onerror event and looks like:

<error status="code" msg="message"/>

ontimeout

raised when the client hasn't heard a message or heartbeat in the timeout interval. The connection is assumed to be closed if this event is raised.

ondisconnect

raised right after the persistent connection is closed. If the connect() method has been called more than once, calling disconnect() once will not trigger this event until the connection count reaches zero.

onuserdisconnect

raised whenever another client disconnects. The connection must have been configured with senduserdisconnect="true".

The same events are raised by the connection manager.

5. Agents

An agent is a back-end server that pushes and receives client data. The OpenLaszlo Server proxies the data between agents and clients. All communication between agent and the server is done using HTTP. Only white-listed agent IPs are allowed, which can be configured using the "connection-agent-ip" option.

<option name="connection-agent-ip">
  <allow>
    <pattern>127.0.0.1</pattern>
  </allow>
</option>

Agents are associated with an url and one or more connection groups. An agent can't send or receive data outside of its own group(s). A connection group is a set of connected applications. Messages sent to a group can be received by applications and other agents registered to that group. An application with an unspecified group name has its webapp path as its group. The OpenLaszlo Server must receive an agent's url and group with each request for validation.

Agents are configured in the application as child elements of the connection. For example:

<connection group="dashboard">
  <agent url="http://127.0.0.1:8080/server-api/History"/>
  <agent url="http://info.com/StockTicker"/>
</connection>

Each declared agent inherits its group from the connection and requires an url, which serves as the agent's unique identifier and its location.

5.1. Receiving data

The OpenLaszlo Server sends data to an agent using the "xml" query parameter. An agent must be able to accept that parameter and parse its contents. Agents designed only to push information can choose to ignore this.

5.2. Sending data

Agents can use the "agentmessage" request type to send data to clients and/or other agents. In addition to the validation parameters, "agentmessage" expects:

to

list of names to send data; use * for everyone

dset

destination dataset; this can be ignored by other agents

msg

arbitrary xml; this parameter is allowed to be empty

range

one of "all", "user", "agent"; if null, default is "all"

The query string to OpenLaszlo Server should look something like:

lzt=agentmessage&url=http://info.com/StockTicker&group=business&to=*&dset=portfolio&msg=<url-encoded><stock-list>
  <stock name="BACL" price="50.0"/><stock name="BGRT" price="38.0"/></stock-list></url-encoded>&range=user

The response from the server will look like:

<lps>
  <status>200</status>
  <message>MESG</message>
</lps>

where MESG is either "message sent" or "no one specified connected (range: [all|user|angent])".

5.3. Getting a list of connected users

The "agentlist" OpenLaszlo request type sends back a list of connected clients and expects only the "users" query parameter, which can be a list of names or *. The query should look something like:

lzt=agentlist&name=stock&password=secret&group=business&users=*

The response from the server will look like:

<lps>
  <status>200</status>
  <message>ok</message>
  <body>
    <list>
      <user name="name1">
      <user name="name2">
      <user name="name3">
      ...
    </list>
  </body>
</lps>

6. Authentication

The OpenLaszlo Server authenticates all server connection requests to ensure that they are not spoofed. Server connection requests are made through function calls in LzConnection and LzConnectionDatasource. They are the connect function, the disconnect function, any of the send message functions, and the get list of connected users function.

The server uses an authenticator to validate each server connection request. You can tell the server what authenticator to use for your connection using the connection tag's authenticator attribute. Parameters for an authenticator can be passed through the authparam attribute. The string has to be in query string format.

<connection authenticator="com.mycompany.Security" 
            authparam="usr=myusr&amp;pwd=mypwd"/>

Note that &amp; was used to XML escape the ampersand. Be sure to URL encode values that contain & or = to %26 and %3D, respectively.

If no authenticator is defined for the connection, the server uses the HTTPAuthentication authenticator. HTTPAuthentication validates a connection by using an application's session cookie. It proxies the cookie to a default security server at http://<lps-host>:<lps-port>/WEBAPP/AuthenticationServlet. You can specify a different security server by passing an url parameter:

<connection authenticator="org.openlaszlo.auth.HTTPAuthentication"
            authparam="url=http://other.host.com/MySecurityServer"/>

If the authentication was successful, the security server should return HTTPAuthentication a username, which, in turn, is returned to the OpenLaszlo Server. You can change the default authenticator with the connection.default.authenticator property. See the "Deployer's Guide" for more on how to configure your server and HTTPAuthentication for what the XML response format of authentication servers should look like.

6.1. AuthenticationServlet

The AuthenticationServlet is the default authentication server used by HTTPAuthentication. You can find the source code for this servlet in WEB-INF/classes/AuthenticationServlet.java.

AuthenticationServlet provides request types for login (create session), logout (remove session), and getusername (validate session) as described below in HTTPAuthentication. During intialization, AuthenticationServlet reads a list of usernames and passwords from WEB-INF/lps/config/lzusers.xml to use for login validation. The format of the file looks like:

<users>
  <user name="adam" password="adam"/>
  <user name="bret" password="bret"/>
  ...
</users>

6.2. Skipping authentication

You can turn off server authentication by using the anonymous authenticator.

<connection authenticator="anonymous"/>

The anonymous authenticator assigns your connection a default username of user. You can change the default value through the connection.none-authenticator.username property. Optionally, you can pass it a usr parameter with the name of your choice. For example, this sets the connection's username to sam:

<connection authenticator="anonymous" authparam="usr=sam"/>

6.3. Implementing an authenticator

Connection authenticators are server-side java objects that implement the org.openlaszlo.servlets.Authentication interface. The server validates a request if and only if an authenticator returns a username. Four functions must be implemented:

void init(Properties prop);

String getUsername(HttpServletRequest req, HttpServletResponse res, 
                   HashMap param) 

int login(HttpServletRequest req, HttpServletResponse res,
          HashMap param, StringBuffer xmlResponse)

int logout(HttpServletRequest req, HttpServletResponse res,
           HashMap param, StringBuffer xmlResponse)

The init() function is called right after an authenticator is instantiated by the OpenLaszlo Server. Parameters from the lps.properties file (located in the WEB-INF/lps/config server directory) are passed into init().

The server uses getUsername() to authenticate each server connection request. A value of null is assumed to mean that the request is invalid. Any other string value, including an empty string, is considered ok.

Both login() and logout() are used by clients to session and unsession the application, respectively.

6.3.1. org.openlaszlo.plugins.HTTPAuthentication

This class makes HTTP requests to back-end authentication servers to session/unsession applications and validate connection requests. Required client cookies are proxied to authentication servers. If set by the authentication server, response headers are proxied back to the client. HTTPAuthentication expects the authentication server to handle login, logout, and getusername request types.

Authentication servers are expected to return an XML response that looks like:

<authentication>
  <response type="login|logout|getusernamey">
    <status code="NUMBER" msg="MESSAGE"/>
    [<username>name</username>]
  </response>
</authentication>

If a status code doesn't exist or isn't 0, the request is assumed to have failed.

6.3.1.1. Login

For sessioning, the login query should look like ?rt=login&usr=userame&pwd=secret.

A successful login will look like:

<authentication>
   <response type="login">
     <status code="0" msg="ok"/>
     <username>username</username>
   </response>
</authentication>

A failed login:

<authentication>
  <response type="login">
    <status code="3" msg="invalid"/>
  </response>
</authentication>

6.3.1.2. Logout

The authentication server must be able to accept the rt=logout parameter for unsessioning.

A successful logout will look like:

<authentication>
  <response type="logout">
    <status code="0" msg="ok"/>
  </response>
</authentication>

A failed logout:

<authentication>
  <response type="logout">
    <status code="4" msg="invalid session"/>
  </response>
</authentication>

6.3.1.3. Getusername

For validation, the authentication server must accept the rt=getusername parameter.

If the client has a valid session, the server should return:

<authentication>
  <response type="getusername">
    <status code="0" msg="ok"/>
    <username>username</username>
  </response>
</authentication>

If the client has an invalid session, the server should return:

 <authentication>
  <response type="getusername">
    <status code="4" msg="invalid session"/>
  </response>
 </authentication> 

6.3.2. org.openlaszlo.servlets.NullAuthentication

Using NullAuthentication is similar to saying:

<connection authenticator="anonymous"/>

However, NullAuthentication allows you to name your connection:

  <connection authenticator="org.openlaszlo.servlets.NullAuthentication"
              authparam="usr=lauren"/>

This connection will be named lauren.

7. Appendix

7.1. Glossary

agent: a back-end server that pushes data to clients and/or other agents, and vice-versa.

application: an LZX program.

authenticator: the server-side java object that authenticates server connection requests.

connection: (see persistent connection)

connection datasource: See LzConnectionDatasource in the LZX Reference.

connection manager: a handler that multiplexes all incoming connection data through a connection datasource into a dataset. See LzConnection in the LZX Reference.

connection session id: the unique identifier of a persistent connection.

server connection requests: LzConnection and LzConnectionDatasource function calls that make requests to the server. They are connect(), disconnect(), sendMessage(), sendXML(), sendUserXML(), and sendAgentXML().

group: a set of applications that can push messages to each other.

heartbeat: a server to client ping to verify that a connection is still alive.

persistent connection: a one-way pipeline from server to client that allows an application to receive asynchronous data.

session id: (see connection session id)

username: a connection name for other applications to target. This value may not be unique among different connections.

7.2. Demo applications

  • Chat (demos/chat/chat.lzx)

  • Dashboard (demos/dashboard/dashboard.lzx)