Simplifying maven release process with nexus staging and maven release plugin for WSO2 git repositories

WSO2 Carbon Kernel is a base platform for all the products of WSO2. With the new version of carbon kernel – 4.3.0, WSO2 has moved to git based source code configuration management (SCM). All the code base is now moved to github under wso2 user account.

When moving to github, the current code base which was at svn had to be re-factored into separate individual repositories. The reason was, the platform repository at svn had the complete code base of all the products together. Release of individual products of current svn based model was based on chunks. A chunk release consist of one or more product release together. So with a new chunk release, the individual components that goes into the those products in that chunk release was branched internally using directory structures and then released using separate pom files which groups all the modules (service-stubs, components, features) together.

But with git, everything is based on branches. So the above branching strategy using directories will not work. Due to this, the chunk based release has to be changed and the platform repository was broke into multiple individual git repositories based on grouping common components. For example, carbon-deployment repository has grouped all components (including service stubs, components and features) related to deployment of artifacts (webapps, axis2services). Now these individual repositories can be released separately and then the products depending on these repositories will be released.

An example is WSO2 Application Server where it depends on the following individual repositories.

For the first release of WSO2 Application Server in git, all the above repositories has to be released before. But this is needed only for the very first release from git. The subsequent releases of WSO2 Application Server can depend on already released versions of these repositories. These dependent repositories are known as upstream projects of WSO2 Application Server product. This is the same for other WSO2 Product based repositories as-well.

Apart from the above, moving to git also has some added advantages

  1. Better management for each git repositories (based on branches)
  2. External contribution can be made easy with better code visibility.
  3. Use of tools like maven-release-plugin and nexus staging repositories can be made easy

With the git based model, the release can be made easy using maven-release-plugin and nexus staging repository. More info about nexus staging and repository management can be read from here.

The following are some common guidelines when doing a release. This guideline focus on carbon4-kernel git project as the example project.

The Tools involved with the following guide lines are

Task – 01

Create a “Repository Target” in http://maven.wso2.org/nexus/ that matches the groupID of the project and add a “Pattern Expression” for it. This pattern expression is used by nexus to automatically determine the staging profile.
Eg: For carbon4-kernel project, below are the values used when creating a “Repository Target”
Name : org.wso2.carbon
Repository Type : Maven2
Pattern Expression : .*/org/wso2/carbon/.*

The main reason for creating separate staging profile and repository target is, for nexus to uniquely identify artifacts belonging to a staging profile, it uses “groupID” of the artifacts. Nexus uses pattern matching for this purpose as above.

Task – 02

Create a nexus “Staging Profile” for the project in http://maven.wso2.org/nexus/, if not already created. The name of profile should match the project groupID.

Eg: For carbon4-kernel project, the name of the profile should be : org.wso2.carbon. As the “Repository Target” for this profile, select the one which was created during the first step above. “Releases” repository at http://maven.wso2.org/nexus/content/repositories/releases/, should be selected as the “Release Repository” for all staging profiles.

Task – 03

Update the project parent pom with the correct SCM configuration.

 
<scm>   
    <url>https://github.com/wso2/carbon4-kernel.git</url>
    <developerConnection>scm:git:https://github.com/wso2/carbon4-kernel.git</developerConnection>
    <connection>scm:git:https://github.com/wso2/carbon4-kernel.git</connection>
    <tag>HEAD</tag>
</scm>

Task – 04

Update project parent pom with the following plugins.

  • Updating the project root pom with WSO2 Master parent pom like below. This pom holds all the common things that are used in almost all the repositories such as distributionManagement, pluginManagement, repositories, pluginRepositories, etc.
 
<parent>
    	<groupId>org.wso2</groupId>
    	<artifactId>wso2</artifactId>
    	<version>1</version>
</parent>
  • Add the following plugins to your build section. The versions of these will be inherited from the above parent pom.
    
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-release-plugin</artifactId>
    <configuration>
        <preparationGoals>clean install</preparationGoals>
    </configuration>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-deploy-plugin</artifactId>
</plugin>

Note : You can add the autoVersionSubmodules configuration parameter to release plugin configuration section which will automatically version the sub modules. But please note that this will cause issues with versioning, if your project has a orbit sub module. This is because for orbit modules, we follow different versioning convention.

Task – 05

Update project distribution management release repository with the following. This step is not mandatory, if the repositories section is inherited from WSO2 Parent Pom.

<repository>
    <id>nexus-releases</id>
    <name>WSO2 Release Distribution Repository</name>       
    <url>http://maven.wso2.org/nexus/service/local/staging/deploy/maven2/</url>
</repository>

Task – 06

Add a server config element in the maven configuration ($MVN_HOME/conf/settings.xml) for the nexus-releases server config above. This is the nexus user credentials which is used for the remote artifact deployment.

<server>
    <id>nexus-releases</id>
    <username>username</username>
    <password>password</password>
</server>

Note : For the above step, you can request WSO2 Infra to create a user for the project in nexus.

Task – 07

Add another server config element that stores the SCM related credentials. This is an optional step, but will be useful to hide your SCM credentials when using mvn-release-plugin.

<server>
    <id>scm-server</id>
    <username>username</username>
    <password>password</password>
</server>

After adding the above, you have to update your project parent pom properties section with the property : “project.scm.id” like below.

<properties>
    <project.scm.id>scm-server</project.scm.id>
</properties>

Task – 08

Create a git release branch from the master. The branch name would be – release-<release-version>

git checkout -b release-<release-version> master

Task – 09

Maven release plugin does not update some properties that we use such as osgi import and export versions. These properties also have “SNAPSHOT” part in it. This has to be manually updated before the performing the release preparation command.
Also make sure that the project does not have any SNAPSHOT dependencies and update those with released versions. If there are any unreleased SNAPSHOT dependencies, we will have to release them separately. This will anyway be checked by the release plugin during release:prepare stage.

To test the above, we can use the “dryRun” option with maven release plugin.

Task – 10

Issue the following release prepare command.

mvn release:clean release:prepare

Note : Please use an appropriate user name for the maven build. The build artifacts will have this username in its MANIFEST file.

Give appropriate values for release, development, and tag versions like below when prompted for the release preparation command.

[INFO] Checking dependencies and plugins for snapshots …
What is the release version for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) 4.3.0: : 4.3.0
What is SCM release tag or label for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) carbon-4.3.0: : 4.3.0
What is the new development version for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) 4.3.1-SNAPSHOT: : 4.4.0-SNAPSHOT

Task – 11

Issue the release perform command as below
mvn release:perform

Task – 12

Once the above process succeeds, the artifacts will be deployed to a staging repo. A newly created staging repo is not automatically closed after artifacts are uploaded. This can be done by the release-manager by logging into the nexus UI and manually closing the repo. Once a staging repo is closed, it will become available for public access.
Eg :
http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-037/

Task – 13

On a failure scenario, the prepared release process can be rolled back which will revert all the commits made during preparation process using the following command.
mvn release:rollback

Task – 14

With the staging repo in effect, a release candidate VOTE should be called on the dev@wso2.org with the following template. This VOTE is essential for a product release. For other projects, this is optional.

[VOTE] Release <Project Name> <Project Version> <RC #>

This is the release candidate of Eg : WSO2 Carbon Kernel 4.3.0 rc1
This release fixes the following issues:
Please download, test and vote. Please refer the release verification guide for detailed information on verifying this release.

Source & binary distribution files:

Maven staging repo: Eg: http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-1000/

The tag to be voted upon: Eg: https://github.com/wso2/carbon4-kernel/releases/tag/carbon4-kernel-4.3.0-rc1

Release verification guide: If any

[ ] Broken - do not release (explain why)
[ ] Stable - go ahead and release

Task – 15

A release VOTE is kept open for 72 hours for the developers to test the artifacts and then vote. A vote is considered as passed, when it has at least 3 binding +1 votes and no -1 votes. Once the release vote passes (the artifacts are tested and verified), the staging repo can be released which will make the artifacts available in the public maven repo. This is again should be done by the release manager. The released artifacts will be available in the WSO2 Releases maven repository at : http://maven.wso2.org/nexus/content/repositories/releases/

Task  – 16

If the vote failed, then the staging repository should be dropped and the changes for the release branch should be reverted and this process should be started again from Task #9 after fixing the issues mentioned during the vote.

Advertisements

About kishanthan

I’m currently working as a Software Engineer at WSO2, an open source software company. I hold an Engineering degree, majoring in Computer Science & Engineering field, from University of Moratuwa, Sri Lanka.
This entry was posted in Carbon, Maven, WSO2 and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s