Chapter 11. Clustering Configuration

Julien Viet

This section covers configuring JBoss Portal to function in a clustered environment.

11.1. Introduction

JBoss Portal leverages various clustered services that are found in JBoss Application Server. This section briefly details how each is leveraged:

  • JBoss Cache: Used to replicate data among the different hibernate session factories that are deployed in each node of the cluster.

  • JBoss HA Singleton:

    1. Used to make the deployer a singleton on the cluster, in order to avoid concurrency issues when deploying the various *-object.xml files. Without that, each node would try to create the same objects in the database when it deploys an archive containing such descriptors.
    2. Used with JCR. The jackrabbit server is not able to run in a cluster by itself, therefore we make a singleton on the cluster. This provides HA-CMS, which is similar to the current HA JBossMQ provided in JBoss AS.

  • HA-JNDI: Used to replicate a proxy that will talk to the HA CMS on the cluster.

  • Http Session Replication: Used to replicate the portal and the portlet sessions.

  • JBoss SSO: Used to replicate the user identity, an authenticated user does not have to login again when failover occurs.

Note

JBoss Clustering details can be found in the Wiki or the clustering documentation.

11.2. Considerations

When you want to run JBoss Portal on a cluster there are a few things to keep in mind:

  • Deploy the portal under the all application server configuration as it contains the clustering services that JBoss Portal leverages.
  • All the portal instances have to use the same datasource : the database is used to store the portal persistent state like pages. If you don't use a shared database then you will have consitency problems.

11.3. JBoss Portal Clustered Services

11.3.1. Portal Session Replication

The portal associates with each user an http session in order to keep specific objects such as:

  • Navigational state : this is mainly the state of the different portlet windows that the user will manipulate during its interactions with the portal. For instance a maximized portlet window with specific render parameters.
  • WSRP objects : the WSRP protocol can require to provide specific cookies during interactions with a remote portlet.

Replicating the portal session ensures that this state will be kept in sync on the cluster, e.g he will see the portlet window he uses exactly the same on every node. The activation of the portal session replication is made through the configuration of the web application that is the main entry point of the portal. This setting is available in the file jboss-portal.sar/portal-server.war/WEB-INF/web.xml

<web-app>
   <description>JBoss Portal</description>
   <!-- Comment/Uncomment to enable portal session replication -->
   <distributable/>
   ...
</web-app>

11.3.2. Hibernate clustering

JBoss Portal leverages hibernate for its database access. In order to improve performances it uses the caching features provided by hibernate. On a cluster the cache needs to be replicated in order to avoid state inconsistencies. Hibernate is configured with JBoss Cache which performs that synchronization transparently. Therefore the different hibernate services must be configured to use JBoss Cache. The following hibernate configurations needs to use a replicated JBoss Cache :

  • jboss-portal.sar/conf/hibernate/user/hibernate.cfg.xml
  • jboss-portal.sar/conf/hibernate/instances/hibernate.cfg.xml
  • jboss-portal.sar/conf/hibernate/portal/hibernate.cfg.xml
  • jboss-portal.sar/conf/hibernate/portlet/hibernate.cfg.xml

The cache configuration should look like :

<!--
   | Uncomment in clustered mode : use transactional replicated cache
   -->
   <property name="cache.provider_class">org.jboss.portal.core.hibernate.JMXTreeCacheProvider
   </property>
   <property name="cache.object_name">portal:service=TreeCacheProvider,type=hibernate
   </property>

<!--
   | Comment in clustered mode
   <property name="cache.provider_configuration_file_resource_path">
   conf/hibernate/instance/ehcache.xml</property>
   <property name="cache.provider_class">org.hibernate.cache.EhCacheProvider</property>
-->

Also we need to ensure that the cache is deployed by having in the file jboss-portal.sar/META-INF/jboss-service.xml the cache service uncommented :

<!--
   | Uncomment in clustered mode : replicated cache for hibernate
   -->
   <mbean
   code="org.jboss.cache.TreeCache"
   name="portal:service=TreeCache,type=hibernate">
   <depends>jboss:service=Naming</depends>
   <depends>jboss:service=TransactionManager</depends>
   <attribute name="TransactionManagerLookupClass">
   org.jboss.cache.JBossTransactionManagerLookup</attribute>
   <attribute name="IsolationLevel">REPEATABLE_READ</attribute>
   <attribute name="CacheMode">REPL_SYNC</attribute>
   <attribute name="ClusterName">portal.hibernate</attribute>
   </mbean>

   <mbean
   code="org.jboss.portal.core.hibernate.JBossTreeCacheProvider"
   name="portal:service=TreeCacheProvider,type=hibernate">
   <depends optional-attribute-name="CacheName">portal:service=TreeCache,type=hibernate
   </depends>
   </mbean>

More information can be found here.

11.3.3. Identity clustering

JBoss Portal leverages the servlet container authentication for its own authentication mechanism. When the user is authenticated on one particular node he will have to reauthenticate again if he use another node of the cluster (during a failover for instance). This is valid only for the FORM based authentication which is the default form of authentication that JBoss Portal uses. Fortunately JBoss provides transparent reauthentication of the user called JBoss clustered SSO. Its configuration is in the file $JBOSS_HOME/server/all/deploy/jbossweb-tomcat55.sar/server.xml and the clustered sso valve shall be uncommented

<Valve className="org.jboss.web.tomcat.tc5.sso.ClusteredSingleSignOn" />

More information can be found here.

11.3.4. CMS clustering

The CMS backend storage relies on the Apache Jackrabbit project. Jackrabbit does not support clustering out of the box. So the portal run the Jackrabbit servicey on one node of the cluster using the HA-Singleton technology. The file jboss-portal.sar/portal-cms.sar/META-INF/jboss-service.xml contains the configuration. We will not reproduce it in this documentation as the changes are quite complex and numerous. Access from all nodes of the cluster is provided by a proxy bound in HA-JNDI. In order to avoid any bottleneck JBoss Cache is leveraged to cache CMS content cluster wide.

11.4. Setup

We are going to outline how to setup a two node cluster on the same machine in order to test JBoss Portal HA. The only missing part from the full fledged setup is the addition of a load balancer in front of Tomcat. However a lot of documentation exist on the subject. A detailed step by step setup of Apache and mod_jk is available from the JBoss Wiki.

As we need two application servers running at the same time, we must avoid any conflict. For instance we will need Tomcat to bind its socket on two different ports otherwise a network conflict will occur. We will leverage the service binding manager this chapter of the JBoss AS documentation.

The first step is to copy the all configuration of JBoss into two separate configurations that we name ports-01 and ports-02 :

>cd $JBOSS_HOME/server
>cp -r all ports-01
>cp -r all ports-02

Edit the file $JBOSS_HOME/server/ports-01/conf/jboss-service.xml and uncomment the service binding manager :

<mbean code="org.jboss.services.binding.ServiceBindingManager"
   name="jboss.system:service=ServiceBindingManager">
   <attribute name="ServerName">ports-01</attribute>
   <attribute name="StoreURL">
   ${jboss.home.url}/docs/examples/binding-manager/sample-bindings.xml</attribute>
   <attribute name="StoreFactoryClassName">org.jboss.services.binding.XMLServicesStoreFactory</attribute>
</mbean>

Edit the file $JBOSS_HOME/server/ports-02/conf/jboss-service.xml, uncomment the service binding manager and change the value ports-01 into ports-02:

<mbean code="org.jboss.services.binding.ServiceBindingManager"
   name="jboss.system:service=ServiceBindingManager">
   <attribute name="ServerName">ports-02</attribute>
   <attribute name="StoreURL">
   ${jboss.home.url}/docs/examples/binding-manager/sample-bindings.xml</attribute>
   <attribute name="StoreFactoryClassName">
   org.jboss.services.binding.XMLServicesStoreFactory</attribute>
</mbean>

Setup a database that will be shared by the two nodes and obviously we cannot use an embedded database. For instance using postgresql we would copy the file portal-postgresql-ds.xml into $JBOSS_HOME/server/ports-01/deploy and $JBOSS_HOME/server/ports-02/deploy.

Copy JBoss Portal HA to the deploy directory of the two configurations.

JBoss Cache Configuration Note : To improve CMS performance JBoss Cache is leveraged to cache the content cluster wide. We recommend that you use the following version of JBoss Cache for best performance:

  • JBoss Cache 1.4.0.SP1 and above
  • JGroups 2.2.7 or 2.2.8

When building from source the following command: {core}/build.xml deploy-ha automatically upgrades your JBoss Cache version.

Alternative: If upgrading your JBoss Cache version is not an option, the following configuration change is needed in the jboss-portal-ha.sar/portal-cms.sar/META-INF/jboss-service.xml. Replace the following configuration in the cms.pm.cache:service=TreeCache Mbean:

<!--
   Configuring the PortalCMSCacheLoader
   CacheLoader configuration for 1.4.0
-->
<attribute name="CacheLoaderConfiguration">
   <config>
       <passivation>false</passivation>
       <preload></preload>
       <shared>false</shared>
       <cacheloader>
           <class>org.jboss.portal.cms.hibernate.state.PortalCMSCacheLoader</class>
           <properties></properties>
           <async>false</async>
           <fetchPersistentState>false</fetchPersistentState>
           <ignoreModifications>false</ignoreModifications>
       </cacheloader>
   </config>
</attribute>
         	  

with the following configuration:

<!--
   Configuring the PortalCMSCacheLoader
   CacheLoader configuratoon for 1.2.4SP2
-->
<attribute name="CacheLoaderClass">org.jboss.portal.cms.hibernate.state.PortalCMSCacheLoader
</attribute>
<attribute name="CacheLoaderConfig" replace="false"></attribute>
<attribute name="CacheLoaderPassivation">false</attribute>
<attribute name="CacheLoaderPreload"></attribute>
<attribute name="CacheLoaderShared">false</attribute>
<attribute name="CacheLoaderFetchTransientState">false</attribute>
<attribute name="CacheLoaderFetchPersistentState">false</attribute>
<attribute name="CacheLoaderAsynchronous">false</attribute>        

Finally we can start both servers, open two shells and execute :

>cd $JBOSS_HOME/bin
>./run.sh -c ports-01

>cd $JBOSS_HOME/bin
>./run.sh -c ports-02

11.5. Portlet Session Replication

Web containers offer the capability to replicate sessions of web applications. In the context of a portal using portlets the use case is different. The portal itself is a web application that benefits of web application session replication. We have to make the distinction between local or remote portlets :

  • Local portlets are web applications deployed in the same virtual machine as the portal web application. At runtime the access to a portlet is done using the mechanism of request dispatching. The portlet session is actually a mere wrapper of the underlying http session of the web application in which the portlet is deployed.
  • Remote portlets are accessed using a web service, we will not cover the replication in this chapter.

The servlet specification is very loose on the subject of replication and does not state anything about the replication of sessions during a dispatched request. JBoss Portal offers a portlet session replication mechanism that leverages the usage of the portal session instead which has several advantages

  • Replicate only the portlet that requires it.
  • Portal session replication is just web application replication and is very standard.

There are also some limitation such has you can only replicate portlet scoped attributes of a portlet session. It means that any attribute scoped using application scope are not replicated.

11.5.1. JBoss Portal configuration

The mandatory step to make JBoss Portal able to replicate portlet sessions is to configure the portal web application to be distributed which is explained in Section 11.3.1, “Portal Session Replication”

11.5.2. Portlet configuration

In order to activate portlet session replication you need to

  • Add in the /WEB-INF/web.xml file of your web application a specific listener class
  • Configure your portlet to be distributed in the /WEB-INF/jboss-portlet.xml file.

<web-app>
   ...
   <listener>
      <listener-class> org.jboss.portal.portlet.session.SessionListener </listener-class>
   </listener>
   ...
</web-app>

Example of web.xml file of your web application

<portlet-app>
   ...
   <portlet>
      <portlet-name>YourPortlet</portlet-name>
      ...
      <session-config>
         <distributed>true</distributed>
      </session-config>
      ...
   </portlet>
   ...
</portlet-app>

Configure YourPortlet to be replicated in jboss-portlet.xml

11.5.3. Limitations

As we noted above there are advantages as well as limitations to the clustering configuration

  • You can only replicate portlet scoped attributes of a portlet. The main reason of this is to keep consistency with the session state. If accessing a portlet would trigger replication of application scoped attribute during the rendering of a page then another portlet on the same page could use this attribute for generating its markup. Then the state seen by this second portlet would not be the same according to the order in which the portlets of this page are rendered.
  • Mutable objects need an explicit call to setAttribute(String name, Object value) on the portlet session object in order to trigger replication by the container.

public void processAction(ActionRequest req, ActionResponse resp)
               throws PortletException, IOException
{
   ...
   if ("addItem".equals(action))
   {
      PortletSession session = req.getPortletSession();
      ShoppingCart cart = (PortletSession)session.getAttribute("cart");
      cart.addItem(item);

      // Perform an explicit set in order to signal to the container that the object
      // state has changed
      session.setAttribute("cart", cart);
   }
   ...
}