Session replication of Applications in WSO2 Application Server

Introduction
The Apache Tomcat[1] is mainly used as a servlet container in WSO2 Carbon[2] based servers. It runs as embedded mode in all servers. It also holds some useful features in it, such as ability to deploy webapps, etc. The WSO2 Application Server[3] uses this feature and provide an easy way for users to host and manage their webapps.

There are some other useful features that are available in a standalone tomcat for webapp deployment and management. For example, clustering and session replication support for webapps is one of it. This feature is also available with embedded mode of tomcat as-well. But when using an embedded mode of tomcats in you servers, this is not a straight forward task to get this work in your own environment like what we get from a standalone tomcat instance. For example, enabling clustering can be done by making changes to server descriptor (server.xml) file. But in embedded mode of tomcat, a server descriptor file will not available. So to cater these facilities, there are some custom changes made to the way tomcat is embedded in WSO2 Carbon 4.0.0 based servers.

Support for server descriptor file(server.xml)

From WSO2 Application Server 5.0.0 onwards, embedded tomcat in WSO2 carbon kernel includes support for server descriptor file (server.xml). This feature was not there in earlier releases as the same functionality was done programatically in carbon kernel. But this had some limitations on getting the full features supported by tomcat such as creating new containers, new hosts for a server etc. So by the addition of this, we can get the full support of tomcat as like what we get in a standalone version.

The server descriptor file allows us to customize the tomcat server configurations [4]. In this file, the top level component is Server. For a tomcat instance, we can have at-most one server component. The next level component is Service. There can be one or more Service components for a server component. We can define one or more containers, such as Context, Engine, Host and Cluster, in each of the Service component. For containers like Engine, Host and Cluster, there can be multiple sub components defined. A tomcat valve is one of such component. These valves comes is handy, when you want to work with incoming request, before the request reaches the webapp. The valves can be inserted into the request processing pipe line of the tomcat engine using server descriptor file. This process also can done programmability.

The main limitation in the earlier releases was, all of the above were done programatically. So if a new component or a container needs to added, some code level changes were mandatory. But with the new changes to the way embedded tomcat is integrated and the support for server descriptor file, this wsa made easy. The server descriptor file is located in {CARBON_HOME}/repository/conf/tomcat directory, with “catalina-server.xml” name.

Clustering and Http Session Replication

A standalone tomcat server in a non-clustered environment does not offer support for fail-over or load balancing. Whenever the server goes down, all the current active sessions become lost. And when the server is restarted, users will have to start again by entering all those lost data to resume their work. This is a limitation found in a standalone tomcat server.

But this is not an issue if the server is a part of a cluster. In that case the server can provide both fail-over and scalability support. So when a server node becomes unavailable, another sever node in the cluster can still serve the requests of users. Also when the load to one server node becomes high, the load can be redirected to another node in the cluster using then load balancing technique. This makes a cluster of tomcat server nodes to be highly available and reliable. Also the cluster environment can scale based on the incoming load by adding new nodes to the cluster. These features from a clustered environment provide users an uninterrupted service.

Http Session replication for webapps is a useful feature in a cluster environment. Tomcat has inbuilt support for clustering and session replication. It uses Tribes[5] as the underlying communication framework to send session data and replication messages across the cluster. The session state in one node is replicated to all the other nodes in the cluster. So that when a different node becomes active, the relevant webapp is loaded with the current active session. The user is unaware of this, as he will continue to do his work without any interruption. The subsequent changes in the session is also will be replicated to every other nodes.

The same feature is also available with a embedded mode of tomcat also. So this can be leveraged and used in WSO2 Application Server as-well. But as explained earlier, this is not a straight forward task.

There are some custom changes required to enable this session replication from Tomcat to WSO2 Application Server. The important change is, WSO2 Application Server already uses Axis2 clustering[6] for web services and in Axis2 clustering, the undelying group communication framework is Tribes. To cat also using this framework for communication. So it is no point having two instance of Tribes framework. The same can be used both Axis2 clustering and tomcat clustering as-well. This change has been implanted and will be available from WSO2 Application Server 5.0.0 onwards.

In tomcat, it uses a valve(ReplicationValve) to initiate the session replication. The cluster sessions are manged by two managers in tomcat. They are BackupManager and DeltaManager. The latter replicates session data to all nodes in the cluster where as the former replicates session data to only one backup node. The other nodes in the cluster knows the location of this backup node.

WSO2 has developed its own TomcatValve to start the session replication process and own implementation of ClusterableSessionManager, which is responsible for handling session for a distributable Web application and sending session replication messages.

Following are the instructions to enable session replication in a WSO2 Application Server cluster. The tomcat server descriptor file in WSO2 Carbon Based servers is catalina-server.xml. This is equivalent to the server.xml file which is found in a standalone instance of tomcat.

  • First enable clustering in Axis2. Follow this guide on how to enable axis2 clustering [7]. We also have to enable clustering for Catalina engine by adding the following in between in catalina-server.xml.
<Cluster className="org.wso2.carbon.core.session.CarbonTomcatSimpleTcpCluster"/>
  • Add the following TomcatValve for which is responsible for starting the replication of session. This should be the first of the Tomcat valves.
<Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve"/>
  • Make the Web application distributable by defining the property “<distributable/>” in web.xml. Then, deploy it in the Application Server. Also is Application Server, there ois support of deploying Jaggery[8] applications as-well. This is also kind of a webapp. But to enable distributable, you have to add
    "distributable":true

    to the jaggery.conf file of the app.

The above steps should also be follwoed for other Application Server nodes in the cluster for which session replication should be enabled. These steps are the standard steps for session replication in a standalone tomcat cluster environment. After following the above steps, the webapps, which are marked as distributable, will have the same session replicated across all nodes in the cluster.

A question may arise that, why do we need to enable axis2 clustering here to work with tomcat webapps session replication? This is because, as I explained earlier, we need a group communication mechanism where it allows us to send /receive messages between the members in a cluster.  Tribes is one such framework which is used in Tomcat Clustering as well. Since we already use tribes with axis2 clustering, it is no point using another implementation of the same framework . Hence we have enable axis2 clustering for the sake of using the underlying tribes framework.

Running the sample

To try out this functionality, we can use a sample which is shipped with WSO2 Application Server distribution. The “Hello World” webapp which is found under samples ({WSO2_AS_HOME}/samples/HelloWorldWebapp), uses the current active http session and just prints the number of access. So when this webapp is deployed in the cluster with distributed mode enabled, we can see that when accessing the webapp on different nodes, the accessed number increases accordingly on each node. This shows that whenever the session is accessed or modified, it gets replicated across all the nodes in the cluster.

References

[1] http://tomcat.apache.org/
[2] http://wso2.com/products/carbon/
[3] http://wso2.com/products/application-server/
[4] http://tomcat.apache.org/tomcat-7.0-doc/config/context.html
[5] http://tomcat.apache.org/tomcat-7.0-doc/tribes/introduction.html
[6] http://axis.apache.org/axis2/java/core/docs/clustering-guide.html
[7] http://wso2.org/project/app-server/4.1.2/docs/appserver-clustering.html
[8] http://jaggeryjs.org/

Advertisements

About kishanthan

I’m currently working as a Software Engineer at WSO2, an open source software company. I hold an Engineering degree, majoring in Computer Science & Engineering field, from University of Moratuwa, Sri Lanka.
This entry was posted in Java, Tomcat, WSO2 and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s