Introducing WSO2 Update Manager (WUM)

WSO2 recently released a tool called WUM (WSO2 Update Manager) with the main goal to provide continuous access to all the fixes and improvements for WSO2 products readily available for all the users. Most importantly, this will make the life easy for most of the first time users of a certain product as they will get all the latest updates for that product instantly, rather than waiting for next release to be available. The product download page also informs users about WUM and how many new updates available for that particular product.

This tool also enables existing users of the products to get all the latest fixes and improvements instantly as and when WSO2 release them.

wum

WUM is a simple command line tool. It is available as installers and as binaries. Currently WUM installers are available for Linux (i586 & x64), Windows (i586 & x64) and MacOSX x64 platforms. You have to download the relevant installers for your OS and install it. The install and uninstall information for each of the installers are also given in the download page. After successfully installing WUM, you have to open a new terminal or cmd window and type “wum” to check and verify the installation. You will see the below output if WUM is successfully installed.

wum-start

Above are the set of commands provided by WUM. Once you have successfully installed WUM, you have to first initialize it using the “wum init” command. Please note that to use WUM and its commands, you must have a registered account with WSO2. After successfully initializing, then you can tryout other commands provided. The commands are grouped together based on their usage and for ease of explanation as bellow.

cmd-usage

Additionally each command has its own help content describing what it does . You can find more comprehensive information about each command from this post by Sameera Jayasoma. I recommend to read this post before proceeding.

A cool thing about WUM is that, once you update a product using the “wum update” command, you will be notified via an email about the new updates that were installed onto your product distribution. This email notification contains important information about the updates that were installed, instruction on the fix added with each update, security updates that were installed, etc. This email will also mention about any configuration changes that were introduced with any of the updates and how to apply them in your environment.

Below is an example email content for a particular product update.

u-summary

Along with the email, additionally, all the updated information is also stored within the updated distribution under “<updated-product-dist-zip-file>/updates” directory as below.

dir-structure

By using WUM, you will find that it is very easy to get new updates and fixes instantly. Since WUM is command line based and dev-ops friendly, you can setup a cron job like mechanism to run the WUM update periodically so that you will get updates on a regular basis. For example, you can setup the cron job to run for every week which will give you the updates that are cumulatively released during that week by WSO2. Once everything in place, you can completely automate this process using some tools like puppet or chef. You cron is setup to get new updates every-week and when new updates are available, there will be a new updated distribution created and using puppet like tool, you can push this new updated distribution to your staging environment for testing before moving to production environment.

Please stay tuned for more information about how you can automate the process on updating products using WUM and what are the things to consider when applying the updated product to your production environment.

Advertisements
Posted in WSO2 | Tagged , , , | Leave a comment

WSO2 Carbon Kernel 5.0.0 Released

The WSO2 Carbon Team have recently announced the new release of WSO2 Carbon Kernel 5.0.0. It is now available to download from here. The source and tag location for this release are available here.

What’s new in 5.0.0?

WSO2 Carbon Kernel 5.0.0 will be the core of the next-generation WSO2 Carbon platform. It is completely rearchitected from the ground up with the latest technologies and patterns. Additionally, the Carbon Kernel is now a very lightweight and thin general-purpose OSGi runtime specializing in hosting servers, providing key functionality for server developers. The result is a streamlined and even more powerful middleware platform than ever before. This kernel will be the base for new carbon platform on which all the products will be built. There are bunch of new products coming up based on this kernel. WSO2 Microservices Server is one of the very first product to be released on top of this kernel.

Below is the high level architecture diagram of carbon kernel.

carbon-kernel-architecture-01

Key Features

How to Contribute

Any suggestions or interested in Carbon Kernel 5.0.0 discussions, you can join the dev@wso2.org or architecture@wso2.org mailing lists. Instructions of how to contribute can be found on the documentation site. To report issues, documentation errors regarding WSO2 Carbon Kernel 5.0.0 through the public issue tracking system.

Posted in WSO2 | Tagged , , , | Leave a comment

OSGi Multi-Tenancy with Regions – Case Study

Proposal

WSO2 Carbon is an OSGi based server framework. One of the core features of the upcoming WSO2 Carbon 5.0 (C5 kernel) is to provide multi-tenacy aspect at server framework level. In previous carbon versions, the multi-tenancy aspect was limited to run-time execution only. In there,  the Axis2 Configuration & Context model to achieve the multi-tenancy where each tenant got its own execution space during run-time. But the OSGi environment was not partitioned for tenants and was visible to all, where a bundle (the library and its packages) installed by a tenant was visible to other tenants as well.

The idea here is to implement Multi-Tenancy at OSGi framework level also, so that each tenant gets its totally isolated run-time environment. One possible approach here is to use OSGi “Regions” concept proposed by Eclipse Virgo, to achieve this with the usage of OSGi framework hooks. A region is a grouping of bundles in an OSGi run-time, which is governed by controls when accessing resources (packages, services) from other regions.

How it will work

This is a high level view of how this will work. This does not focus on any implementation details.

Each tenant gets its own region and there will be a separate “Kernel” region where the core bundles, packages and services will be residing. These will be exposed to tenant regions. We can still limit and filter on what to expose from kernel region. Each tenant region will be isolated from each other. They will not see any events related to bundles or services. The package visibility will also be filtered from other regions, but they can only see from within their region self and kernel region.

Below image is high level view of this concept.

OSGi Multi Tenancy

Below are the overview of the initially planned framework hooks to implement this feature.

  • CarbonRegionResolverHook – manages the package resolve process for requirements from bundles in regions.
  • CarbonRegionBundleFindHook – manages/filters the BundleContext.getBundle lookups from region bundles.
  • CarbonRegionBundleEventHook – manages/filters the bundle’s life-cycle events for regions.
  • CarbonRegionBundleCollisionHook – manages the duplicate bundle resolving in multiple regions. This will facilitate to have same bundles in different regions.
  • CarbonRegionServiceFindHook and CarbonRegionServiceEventHook – manages/filters the service lookup and life-cycle events for regions.
Posted in Carbon, Carbon5, OSGi, WSO2 | Tagged , , , , , , , , | Leave a comment

WSO2 Application Server 5.3.0 Alpha Released !!!

WSO2 Application Server Team has recently done an Alpha release of the latest Application Server Product. This is based on the latest in improved WSO2 Carbon Kernel 4.4.0. This is first release of Application Server product after moving to the git based repositories. It also has added some key new features such as JaveEE 6 Web Profile Support, Websocket 1.1. support, etc. Below is the release note from the WSO2 Application Server Team.

Release Note of WSO2 AS 5.3.0

WSO2 Application Server team is pleased to announce the first alpha release of WSO2 AS 5.3.0. The AS 5.3.0 Alpha distribution is available from this location. This release introduces several major features and improvements.

New Features

  • JavaEE 6 Web Profile support via TomEE 1.7.2 integration
  • Websocket 1.1 API Support as defined by the JSR-356 specification
  • SAML2 Single-Sign-On support for web applications
  • Tomcat Virtual Hosts support
  • HTTP Session Persistence support
  • WS-Discovery support for CXF JAX-WS and JAX-RS services
  • OSGi ServiceLoader Mediator specification support via SPI-Fly
  • Removed the first class support provided for deploying Axis2 AAR Services
  • Removed the data services hosting support from AS. Users can use the WSO2 Data Services Server product to deploy data services.

Existing Key Features

  • Support for Servlet 3, JSP 2.2, EL 2.2, JSTL 1.2 specifications.
  • Full JAX-WS 2.2 and JAX-RS 2.0 Specification support
  • Integration of Jaggery – server side scripting framework
  • Unified Application listing and management UI for WebApps, JAX-WS/RS, Jaggery
  • Multi Tenant support for standalone deployment
  • 100% Apache Tomcat compliance runtime
  • Lazy loading for web applications and services
  • Tooling – Application Server related artifacts can be easily generated using WSO2 Developer Studio
  • Clustering support for High Availability and High Scalability
  • Full support for WS-Security, WS-Trust, WS-Policy and WS-Secure Conversation
  • JMX and Web interface based monitoring and management
  • WS-* and REST support
  • GUI, command line, and IDE based tools for Web service development
  • Equinox P2 based provisioning support
  • WSDL2Java/Java2WSDL/WSDL 1.1, and UI-based try it (invoke any remote Web service)
Issues Fixed in This Release

https://wso2.org/jira/issues/?filter=12252

Known Issues

https://wso2.org/jira/issues/?filter=12253

How You Can Contribute

Reporting Issues

WSO2 encourages users to report issues and your enhancement requests for the WSO2 AS using the AS Public JIRA.

Posted in WSO2 | Tagged , , , | Leave a comment

Things to consider when writing WSO2 Carbon Components and Features

In this post, I’m looking at some important things to consider when writing WSO2 Carbon based components and features. But most of them are common and can be used for other development related activities as-well.

Contents focused in this post are : 

OSGi level best practices.

  • Proper bundle creation using bundle plugin
  • Proper package imports and exports with defined versions (or range)
  • Avoid using Dynamicimport-Package *, why?
  • Avoid using *;resolution:=optional, why?
  • Avoid using Require-Bundle, why?
  • Proper usage if bundle concepts (eg : internal package)

Maven dependency definitions related best practices.

  • Use parent pom for dependencyManagement, pluginManagement.
  • Use property based approach with defining dependency versions
  • Use only required dependencies (remove unused)

Carbon components and features related best practices

  • Front-End/Back-End separated components/features.
  • Properly defining features (including vs importing)

Business functionality level best practices.

  • Follow API and Implementation separation when needed.
  • Use OSGi Declarative Service approach for exposing/reusing other component functionalities when needed.

Introduction

A carbon component is the base unit for all features supported by WSO2 platform such as security, clustering, logging, statistics, and management. This is called the compotenized middleware, which is build using the OSGi – The Dynamic Modular System for Java.

There are quite a number of resources and articles on how to write a new carbon component to fit the requirement at hand. But many users and developers tend to forget the best practices that should be followed when writing a carbon component and then the related carbon feature.

Maintaining a carbon component is easy when it was followed with the best practices during the time it was written. This allows the developers of the carbon platform to troubleshoot errors easily with components. If not it becomes a nightmare to find a fix some issues with components. (Eg : OSGi level package imports/exports with class loading issues).

OSGi level best practices

The OSGi plays a major role with carbon components. Since WSO2 Carbon Platform uses OSGi for the underlying modular layer, it is important to follow these best practices.

Proper bundle manifest headers using bundle plugin

The most important task in building OSGi bundles is to write the MANIFEST.MF file correctly. This is where the package information is defined for bundles. The tool used for this task is the bundle plugin, because manually writing this file is not easy.

The below is a sample bundle plugin configuration for a bundle

<plugins>
 <plugin>
   <groupId>org.apache.felix</groupId>
   <artifactId>maven-bundle-plugin</artifactId>
   <configuration>
     <instructions>
       <Bundle-Vendor>WSO2 Inc</Bundle-Vendor>
       <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
     </instructions>
   </configuration>
 </plugin>
</plugins>

The WSO2 way is to use project.artifactId as the bundle symbolic name and bundle vendor as “WSO2 Inc”. More info on using bundle plugin can be found from this article.

Package import and export with defined versions (or range)

The unit of isolation for bundles(or carbon components) at OSGi level is the packages. Bundles will import packages from other bundles for its usage and export packages to the OSGi environment for other bundles to use. They will also have some private packages, which are not exposed outside, but used within the bundle only.

Here is an example bundle plugin configuration

<plugin>
    <groupId>org.apache.felix</groupId>
    <artifactId>maven-bundle-plugin</artifactId>
    <extensions>true</extensions>
    <configuration>
        <instructions>
            <Bundle-Vendor>WSO2 Inc</Bundle-Vendor>
            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
            <Bundle-Activator>
                 org.wso2.carbon.clustering.internal.CarbonClusterBundleActivator
            </Bundle-Activator>
            <Private-Package>
                org.wso2.carbon.clustering.internal
            </Private-Package>
            <Export-Package>
                !org.wso2.carbon.clustering.internal,
                org.wso2.carbon.clustering.*;version="5.0.0"
            </Export-Package>
            <Import-Package>
                org.slf4j.*;version="${slf4j.logging.import.version.range}",
                org.osgi.framework.*;version="${osgi.framework.import.version.range}",
                org.osgi.service.component.*;version="[1.0.0, 1.1.0)",
                com.hazelcast.core.*;version="[3.0.1, 3.1.0)",
            </Import-Package>
        </instructions>
    </configuration>
</plugin>

Best practices.

  1. Export packages with a version.

This will ensure that the packages you export will have a version at run-time. So that interested parties can import packages using that version. If a version is not specified, then it will be considered as 0.0.0 by the framework. Also if you want to exclude some packages, then you can use the “!”.

  1. Import packages with the version range.

This is needed in order to ensure that you bundle gets wired with the correct bundle at runtime. There can be multiple bundles exporting the same package, but with different versions. If a version range is not specified, your bundle gets wired with the bundle which exports the higher version of the package. This may lead to some issues, unexpected behaviour at runtime, where the packages imported may have different classes, methods or business logic.

When importing packages the best practice is to use the semantic version approach with pattern like com.foo.bar.*;version=”[1.0.0, 1.1.0)”,

This informs that my bundle requires com.foo.bar.* package that is exported in the version range of >=1.0.0 and <1.1.0. This is normal way to ensure that there will be no API changes of the package from a minor version.

The version range for non specified packages are considered like (0.0.0, ). This means that any version that is between 0.0.0 to can get wired with the bundle.

Avoid using Dynamicimport-Package “*”

The <DynamicImport-Package>*</DynamicImport-Package> manifest header is used to allow a bundle to be wired up to packages that may not known in advance. This may look like a easy way to handle package imports, but it tends to break the modular nature of OSGi. The reason is that this will make the class-path search of OSGi Framework a very expensive operation for the packages involved and breaks the concept of versions in OSGi.

This header is only required in a situation where the Class.forName() is used within your bundle. Other than that, it is not recommended to use this header.

Avoid using *;resolution:=optional

The resolution:=optional directive is used in a situation where some packages or rather dependencies are not required for the bundle to resolve. This is ok if some packages are not found in run-time but your bundle needs to resolve. But the directive with “*” will become an issue where the bundle plugin will find all package imports for you and place it in the manifest file, when you have not explicitly mentioned them in import packages section. It will also put some version range for those missing imports based on the dependency version. Sometimes, this will become hard when resolving issues where you bundle gets wired to a different package (bundle) but not the expected one at run-time. You can use this directive in a situation where some packages are actually not needed for your bundle to resolve at run-time. But the same directive with “*” is not recommended, as it may cause bundle wiring issues.

Avoid using Require-Bundle

This is actually an anti pattern in OSGi world, where a complete bundle is added as a dependency to a bundle. The recommended approach is to use package imports/exports from/to bundles.

Proper usage of bundle concepts (eg : internal package)

The internal package (conceptually package name consisting “internal” in any WSO2 carbon components) of a bundle is considered as private to that bundle and used within the bundle only. This is where the BundleActivator class and OSGi Service Holder classes reside. These are private to that bundle and should not be exposed to outside.

For exposing/reusing other component functionalities the recommended approach is to use OSGi Services. The internal framework which supports the dynamic nature for services is the Declarative Services Framework.

Maven dependency definitions related best practices

Since maven is the mostly used tool for the project build management, it is also important to use the best practices with defining pom files and project.

Use parent pom for dependencyManagement, pluginManagement, repository management.

The recommended approach for defining maven build management sections such as dependencyManagement, pluginManagement, repository (both dependency and plugins) is to use the root parent pom. This the parent for all the modules in maven project. Each sub-modules (or child) will refer these. They will not have to define the versions for these when referring them in their poms. The parent pom is the only place to define all the dependency , plugin, versions, and other common properties. This will enable the easy management of the project, where you will need to do minimal change when you’re updating a version. An example pom which follow the above best practices is found here

Use property based approach with defining dependency versions

When a dependency, or a plugin is defined in the management section in the parent pom, it is recommended to use property based approach to define the versions.

Consider the following example

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-server</artifactId>
    <version>${version.jetty.server}</version>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-http</artifactId>
    <version>${version.jetty.server}</version>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-util</artifactId>
    <version>${version.jetty.server}</version>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-io</artifactId>
    <version>${version.jetty.server}</version>
</dependency>

All the above dependencies are using the same version. So as given above, when using the property based approach (${version.jetty.server}), the version can be easily managed. Also it is not required to specify the version for these dependencies in the child modules that refer these.

Use only required dependencies

With dependencies definition there are different types. Some dependencies properly declared and used and some comes from transitive dependencies which is not properly declared in your project.

The recommended way is declare dependencies that are used only. Other non related dependencies can be easily identified using the maven dependency plugin. More info on this can be found from here.

Carbon components and features related best practices

Front-End and Back-End separated components/features. 

When you are developing a new carbon component, in most of the cases you will need to provide the UI front for that component. This is where the users will interact with the functionality you provide with the component. There will be a separate UI component, which talk to the back-end component via http/s (SOAP). This separation of front-end and back-end in a carbon component is known as the FE/BE separation. This is the recommended way in developing new carbon components. The same concept is used when developing the feature for your component, where for the UI related bundles, there will be separate UI feature and for back-end related bundles, there will be a back-end feature and there will be a composite feature which combine these features. More on writing carbon components is available here.

Properly defining features

Naming convention

The first thing to note about when defining new features is the naming convention to follow. A feature should have a descriptive name for the users to identify them uniquely and a feature ID for run-time identification. Consider the following example feature. The highlighted section is self describing about the feature.

<build>
    <plugins>
        <plugin>
            <groupId>org.wso2.maven</groupId>
            <artifactId>carbon-p2-plugin</artifactId>
            <version>${carbon.p2.plugin.version}</version>
            <executions>
                <execution>
                    <id>4-p2-feature-generation</id>
                    <phase>package</phase>
                    <goals>
                        <goal>p2-feature-gen</goal>
                    </goals>
                    <configuration>
                        <id>org.wso2.carbon.cluster.mgt.server</id>
                        <propertiesFile>../../../etc/feature.properties</propertiesFile>
                        <adviceFile>
                            <properties>
                                <propertyDef>org.wso2.carbon.p2.category.type:server</propertyDef>
                                <propertyDef>org.eclipse.equinox.p2.type.group:false</propertyDef>
                            </properties>
                        </adviceFile>
                        <bundles>
                            <bundleDef>org.wso2.carbon:org.wso2.carbon.cluster.mgt.admin</bundleDef>
                        </bundles>
                        <importFeatures>
                            <importFeatureDef>org.wso2.carbon.core.server:${wso2carbon.version}</importFeatureDef>
                        </importFeatures>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Feature categories

The general categorization for carbon features are Server Feature, UI (Console) Feature and Common Feature (This is an optional category). All of these categories should be added to Composite (Aggregate) Feature as an aggregate.

Feature construction

Carbon features are constructed using the carbon-p2 maven plugin. It has a set of instruction to create a feature.

  1. Include bundles
  2. Include features
  3. Import bundles
  4. Include features

The above instructions are used based on the requirement for the feature that you are developing.

Including bundles

This is used when a bundle/jar should be included in the feature as sub part which makes the feature to tightly couple with it. The bundle/jar also gets packed with feature. In the above feature example this is carried using the following element.

<bundles>
	<bundleDef>org.wso2.carbon:org.wso2.carbon.cluster.mgt.admin</bundleDef>
</bundles>

Including features

Same as including bundle, other features can also be included into a feature and gets packed with it. The common example for this is the composite feature, which includes both UI and Server feature as given below.

<includedFeatures>
	<includedFeatureDef>org.wso2.carbon:org.wso2.carbon.cluster.mgt.server.feature</includedFeatureDef>
	<includedFeatureDef>org.wso2.carbon:org.wso2.carbon.cluster.mgt.ui.feature</includedFeatureDef>
</includedFeatures>

Importing bundles

This is used when a bundle is required as a dependency to the feature, but not tightly coupled with it. For example apache ode bundle is required  as a dependency for org.wso2.carbon.bpel.ui.feature.

<importBundles>
	<importBundleDef>org.apache.ode.wso2:ode</importBundleDef>
</importBundles>

Importing Features

Like with importing bundles, this is used when a feature depends on other features for correct behavior of that feature. An example will look like below. This is common approach when your feature requiring some other feature for its operation.

<importFeatures>
	<importFeatureDef>org.wso2.carbon.core:${wso2carbon.version}</importFeatureDef>
</importFeatures>

Best practices with defining features

The important point to note here is that, when to use which of the above instruction. If a feature requires other features as dependencies, the correct way is to use “ImportedFeatures”. This will allow the feature of interest to be loosely coupled with dependent features and will allow updates to new version of the dependent feature suing installation. This is same for bundle imports as well. Instead of importing but including features/bundles in wrong place, will lead to a situation where updates to new version of them is not available and some time will result in installation failure. There are more information on the best practices to follow with features is found in this resource.

Business functionality level best practices.

Follow API and Implementation separation when needed

When writing a solution based on OSGi, it is always best to have a separation between implementation and the related API’s. This allows the users of the API not to get affected for any changes in the implementation. For example consider a situation where the clustering framework is designed using the above pattern. This makes easy for a new clustering implementation to be plugged in based on the API. The user of the clustering service is not aware of the underlying implementation changes and continue to use the clustering service.

Use OSGi Declarative Service (DS) approach when needed.

For exposing/reusing other component functionalities the recommended approach is to use OSGi Services. The internal framework which supports the dynamic nature for services is the Declarative Services Framework. In the below example, we can see how to use DS based approach and the things to consider when using it within carbon framework.

An example DS component

/**
 * @scr.component name="component.manager.core.service.comp" immediate="true"
 * @scr.reference name="provisioning.agent.provider"
 * interface="org.eclipse.equinox.p2.core.IProvisioningAgentProvider"
 * cardinality="1..1" policy="dynamic" bind="setProvisioningAgentProvider"
 * unbind="unsetProvisioningAgentProvider"
 * @scr.reference name="server.config.service" interface="org.wso2.carbon.base.api.ServerConfigurationService"
 * cardinality="1..1" policy="dynamic"  bind="setServerConfigurationService" unbind="unsetServerConfigurationService"
 */
public class ComponentMgtCoreServiceComponent {

    protected void activate(ComponentContext ctxt) {
        ctxt.getBundleContext().registerService(CommandProvider.class.getName(), new ProvCommandProviderExt(), null);       
    }

    protected void setProvisioningAgentProvider(IProvisioningAgentProvider provisioningAgentProvider) {
        ServiceHolder.setProvisioningAgentProvider(provisioningAgentProvider);
    }

    protected void unsetProvisioningAgentProvider(IProvisioningAgentProvider provisioningAgentProvider) {
        ServiceHolder.setProvisioningAgentProvider(null);
    }
    
    protected void setServerConfigurationService(ServerConfigurationService serverConfigService) {
	ServiceHolder.setServerConfigurationService(serverConfigService);
    }

    protected void unsetServerConfigurationService(ServerConfigurationService serverConfigService) {
	ServiceHolder.setServerConfigurationService(null);
    }
}

In the above example, the DS component (component.manager.core.service.comp), is referring two different OSGi service references for it to become active. They are “IProvisioningAgentProvider” and “ServerConfigurationService”. These services are registered by some other bundles/components and when they become available, this component is notified and will be able to acquire the references for them. This process is handled by the underlying OSGi DS framework. When the referring services become available in the run-time, the setter methods of those respective scr.reference will be called.

Use of ServiceHolder approach

The point to note here is that what do we do when this method is called?

The widely known approach for most of the components is storing this references in a “Holder” and then use it when required within the bundle. This is known as the “ServiceHolder” approach. In the above example also we can see that the references are saved in the ServiceHolder within the setter methods. Also when the references become unavailable (since OSGi run-time is highly dynamic, the service references may become available / unavailable any time), the corresponding unsetter methods will be called, in which the references are removed from the ServiceHolder.

The ServiceHolder is considered as the internal part of the bundle and should not be exposed outside of bundle. To achieve this, the ServiceHolder classes are kept within the private package section of the bundle.

Posted in Carbon, OSGi, WSO2 | Tagged , , , , , , , , , | 1 Comment

Simplifying maven release process with nexus staging and maven release plugin for WSO2 git repositories

WSO2 Carbon Kernel is a base platform for all the products of WSO2. With the new version of carbon kernel – 4.3.0, WSO2 has moved to git based source code configuration management (SCM). All the code base is now moved to github under wso2 user account.

When moving to github, the current code base which was at svn had to be re-factored into separate individual repositories. The reason was, the platform repository at svn had the complete code base of all the products together. Release of individual products of current svn based model was based on chunks. A chunk release consist of one or more product release together. So with a new chunk release, the individual components that goes into the those products in that chunk release was branched internally using directory structures and then released using separate pom files which groups all the modules (service-stubs, components, features) together.

But with git, everything is based on branches. So the above branching strategy using directories will not work. Due to this, the chunk based release has to be changed and the platform repository was broke into multiple individual git repositories based on grouping common components. For example, carbon-deployment repository has grouped all components (including service stubs, components and features) related to deployment of artifacts (webapps, axis2services). Now these individual repositories can be released separately and then the products depending on these repositories will be released.

An example is WSO2 Application Server where it depends on the following individual repositories.

For the first release of WSO2 Application Server in git, all the above repositories has to be released before. But this is needed only for the very first release from git. The subsequent releases of WSO2 Application Server can depend on already released versions of these repositories. These dependent repositories are known as upstream projects of WSO2 Application Server product. This is the same for other WSO2 Product based repositories as-well.

Apart from the above, moving to git also has some added advantages

  1. Better management for each git repositories (based on branches)
  2. External contribution can be made easy with better code visibility.
  3. Use of tools like maven-release-plugin and nexus staging repositories can be made easy

With the git based model, the release can be made easy using maven-release-plugin and nexus staging repository. More info about nexus staging and repository management can be read from here.

The following are some common guidelines when doing a release. This guideline focus on carbon4-kernel git project as the example project.

The Tools involved with the following guide lines are

Task – 01

Create a “Repository Target” in http://maven.wso2.org/nexus/ that matches the groupID of the project and add a “Pattern Expression” for it. This pattern expression is used by nexus to automatically determine the staging profile.
Eg: For carbon4-kernel project, below are the values used when creating a “Repository Target”
Name : org.wso2.carbon
Repository Type : Maven2
Pattern Expression : .*/org/wso2/carbon/.*

The main reason for creating separate staging profile and repository target is, for nexus to uniquely identify artifacts belonging to a staging profile, it uses “groupID” of the artifacts. Nexus uses pattern matching for this purpose as above.

Task – 02

Create a nexus “Staging Profile” for the project in http://maven.wso2.org/nexus/, if not already created. The name of profile should match the project groupID.

Eg: For carbon4-kernel project, the name of the profile should be : org.wso2.carbon. As the “Repository Target” for this profile, select the one which was created during the first step above. “Releases” repository at http://maven.wso2.org/nexus/content/repositories/releases/, should be selected as the “Release Repository” for all staging profiles.

Task – 03

Update the project parent pom with the correct SCM configuration.

 
<scm>   
    <url>https://github.com/wso2/carbon4-kernel.git</url>
    <developerConnection>scm:git:https://github.com/wso2/carbon4-kernel.git</developerConnection>
    <connection>scm:git:https://github.com/wso2/carbon4-kernel.git</connection>
    <tag>HEAD</tag>
</scm>

Task – 04

Update project parent pom with the following plugins.

  • Updating the project root pom with WSO2 Master parent pom like below. This pom holds all the common things that are used in almost all the repositories such as distributionManagement, pluginManagement, repositories, pluginRepositories, etc.
 
<parent>
    	<groupId>org.wso2</groupId>
    	<artifactId>wso2</artifactId>
    	<version>1</version>
</parent>
  • Add the following plugins to your build section. The versions of these will be inherited from the above parent pom.
    
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-release-plugin</artifactId>
    <configuration>
        <preparationGoals>clean install</preparationGoals>
    </configuration>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-deploy-plugin</artifactId>
</plugin>

Note : You can add the autoVersionSubmodules configuration parameter to release plugin configuration section which will automatically version the sub modules. But please note that this will cause issues with versioning, if your project has a orbit sub module. This is because for orbit modules, we follow different versioning convention.

Task – 05

Update project distribution management release repository with the following. This step is not mandatory, if the repositories section is inherited from WSO2 Parent Pom.

<repository>
    <id>nexus-releases</id>
    <name>WSO2 Release Distribution Repository</name>       
    <url>http://maven.wso2.org/nexus/service/local/staging/deploy/maven2/</url>
</repository>

Task – 06

Add a server config element in the maven configuration ($MVN_HOME/conf/settings.xml) for the nexus-releases server config above. This is the nexus user credentials which is used for the remote artifact deployment.

<server>
    <id>nexus-releases</id>
    <username>username</username>
    <password>password</password>
</server>

Note : For the above step, you can request WSO2 Infra to create a user for the project in nexus.

Task – 07

Add another server config element that stores the SCM related credentials. This is an optional step, but will be useful to hide your SCM credentials when using mvn-release-plugin.

<server>
    <id>scm-server</id>
    <username>username</username>
    <password>password</password>
</server>

After adding the above, you have to update your project parent pom properties section with the property : “project.scm.id” like below.

<properties>
    <project.scm.id>scm-server</project.scm.id>
</properties>

Task – 08

Create a git release branch from the master. The branch name would be – release-<release-version>

git checkout -b release-<release-version> master

Task – 09

Maven release plugin does not update some properties that we use such as osgi import and export versions. These properties also have “SNAPSHOT” part in it. This has to be manually updated before the performing the release preparation command.
Also make sure that the project does not have any SNAPSHOT dependencies and update those with released versions. If there are any unreleased SNAPSHOT dependencies, we will have to release them separately. This will anyway be checked by the release plugin during release:prepare stage.

To test the above, we can use the “dryRun” option with maven release plugin.

Task – 10

Issue the following release prepare command.

mvn release:clean release:prepare

Note : Please use an appropriate user name for the maven build. The build artifacts will have this username in its MANIFEST file.

Give appropriate values for release, development, and tag versions like below when prompted for the release preparation command.

[INFO] Checking dependencies and plugins for snapshots …
What is the release version for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) 4.3.0: : 4.3.0
What is SCM release tag or label for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) carbon-4.3.0: : 4.3.0
What is the new development version for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) 4.3.1-SNAPSHOT: : 4.4.0-SNAPSHOT

Task – 11

Issue the release perform command as below
mvn release:perform

Task – 12

Once the above process succeeds, the artifacts will be deployed to a staging repo. A newly created staging repo is not automatically closed after artifacts are uploaded. This can be done by the release-manager by logging into the nexus UI and manually closing the repo. Once a staging repo is closed, it will become available for public access.
Eg :
http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-037/

Task – 13

On a failure scenario, the prepared release process can be rolled back which will revert all the commits made during preparation process using the following command.
mvn release:rollback

Task – 14

With the staging repo in effect, a release candidate VOTE should be called on the dev@wso2.org with the following template. This VOTE is essential for a product release. For other projects, this is optional.

[VOTE] Release <Project Name> <Project Version> <RC #>

This is the release candidate of Eg : WSO2 Carbon Kernel 4.3.0 rc1
This release fixes the following issues:
Please download, test and vote. Please refer the release verification guide for detailed information on verifying this release.

Source & binary distribution files:

Maven staging repo: Eg: http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-1000/

The tag to be voted upon: Eg: https://github.com/wso2/carbon4-kernel/releases/tag/carbon4-kernel-4.3.0-rc1

Release verification guide: If any

[ ] Broken - do not release (explain why)
[ ] Stable - go ahead and release

Task – 15

A release VOTE is kept open for 72 hours for the developers to test the artifacts and then vote. A vote is considered as passed, when it has at least 3 binding +1 votes and no -1 votes. Once the release vote passes (the artifacts are tested and verified), the staging repo can be released which will make the artifacts available in the public maven repo. This is again should be done by the release manager. The released artifacts will be available in the WSO2 Releases maven repository at : http://maven.wso2.org/nexus/content/repositories/releases/

Task  – 16

If the vote failed, then the staging repository should be dropped and the changes for the release branch should be reverted and this process should be started again from Task #9 after fixing the issues mentioned during the vote.

Posted in Carbon, Maven, WSO2 | Tagged , , , , , , , | Leave a comment

WSO2 Carbon Kernel 4.3.0 Released

WSO2 Carbon Team has just finished with their new release of WSO2 Carbon Kernel 4.3.0 which is the very first release of WSO2 Carbon based on GitHub.

Here is the release note from WSO2 Carbon Team.

WSO2 Carbon team is pleased announce the release of the Carbon Kernel 4.3.0.

What is WSO2 Carbon

WSO2 Carbon redefines middleware by providing an integrated and componentized middleware platform that adapts to the specific needs of any enterprise IT project – on premise or in the cloud. 100% open source and standards-based, WSO2 Carbon enables developers to rapidly orchestrate business processes, compose applications and develop services using WSO2 Developer Studio and a broad range of business and technical services that integrate with legacy, packaged and SaaS applications.

WSO2 Carbon kernel, the lean, modular, OSGi-based platform, is the base of the WSO2 Carbon platform. It is a composable server architecture which inherits modularity and dynamism from OSGi framework. WSO2 Carbon kernel can be considered as a framework for server development. All the WSO2 products are composed as a collection reusable components running on this kernel. These products/components inherits all the core services provided by Carbon kernel such as Registry/repository, User management, Transports, Caching, Clustering, Logging, Deployment related features.

You can download the released distribution from the product home page : http://wso2.com/products/carbon/

How to Contribute 
What’s New In This Release
  • Simplified logging story with pluggable log provider support.
  • Upgraded versions of Hazelcast, Log4j, BouncyCastle.
  • Improved Composite application support.

Key Features

  • Composable Server Architecture – Provides a modular, light-weight, OSGi-based server development framework.
  • Carbon Application(CApp) deployment support.
  • Multi-Profile Support for Carbon Platform – This enable a single product to run on multiple modes/profiles.
  • Carbon + Tomcat JNDI Context – Provide ability to access both carbon level and tomcat level JNDI resources to applications using a single JNDI context.
  • Distributed Caching and Clustering functionality – Carbon kernel provides a distributed cache and clustering implementation which is based on Hazelcast- a group communication framework
  • Pluggable Transports Framework – This is based on Axis2 transports module.
  • Registry/Repository API- Provide core registry/repository API for component developers.
  • User Management API  – Provides a basic user management API for component developers.
  • Logging – Carbon kernel supports both Java logging as well as Log4j. Logs from both these sources will be aggregated to a single output
  • Pluggable artifact deployer framework – Kernel can be extended to deploy any kind of artifacts such as Web services, Web apps, Business processes, Proxy services, User stores etc.
  • Deployment Synchronization – Provides synchronization of deployed artifacts across a product cluster.
  • Ghost Deployment – Provides a lazy loading mechanism for deployed artifacts</li>
  • Multi-tenancy support – The roots of the multi-tenancy in Carbon platform lies in the Carbon kernel. This feature includes tenant level isolation as well as lazy loading of tenants.
Fixed Issues
Known Issues
Contact Us 

WSO2 Carbon developers can be contacted via the mailing lists:

Reporting Issues
You can use the Carbon JIRA issue tracker to report issues, enhancements and feature requests for WSO2 Carbon.
Posted in WSO2 | Tagged , , , , | 1 Comment