Using Ant and Jenkins to automate the deployment of Java projects from the Azure Toolkit for Eclipse

While the Azure Toolkit for Eclipse greatly simplifies the initial Java project creation and deployment to Azure, there is often a point at which a need arises for automating subsequent deployments without an IDE. Traditionally, automation is achieved by command line tools such as Ant or Maven, or CI systems such as Jenkins. This tutorial describes how to automate the deployment of Java projects to Azure using the Azure Toolkit’s command line Ant support and Jenkins. Many of the concepts presented here can also be applied in other automation scenarios or with other command line tools.

Azure Toolkit’s Use of Ant

Although the Azure Toolkit for Eclipse is an IDE plugin with rich UI functionality, at its core its build and cloud deployment logic is implemented as an Ant extension, independent of the IDE. When you build and publish an Azure deployment project using the IDE (as described in this Hello World tutorial), the output you see in the Eclipse Console window is produced by that Ant extension. This makes the task of running deployment logic outside the IDE very easy.

The Toolkit even gives you a head start by auto-generating a set of short and simple Ant scripts as part of its build and publish process. After you publish an Azure deployment project to the cloud at least once, look for the cloudTools directory under the project directory, where you will see a set of simple scripts, in both Windows .CMD and Linux/Mac .SH formats, which show how to run the build and publish process in Ant:

To catch a glimpse of what’s going on under the hood, take a look at buildAndPublish.sh (Mac/Linux) or buildAndPublish.cmd(Windows). It just has the following simple line in it:

ant -f ../package.xml publish

Assuming the Ant executable is in your global command line path and you are running this command from within the project directory, the above command line is all that is needed to execute that logic. Double-clicking on the script file (or select Open With > System Editor from the context menu) in Eclipse executes the command in the IDE. A system console window with build log output that is identical to the output in Eclipse’s Console view is displayed, that looks something like this:

You can use the buildAndPublish script directly as a shortcut to run this Ant task, or as an educational sample showing how to invoke the Ant task from command line.

Invoking the Ant logic from Jenkins on the same computer as Eclipse

Now that you see what it takes to automate the building and publishing of this deployment via Ant from the command line, let’s take a look at what it would take to invoke this logic from a Jenkins build job. For starters, we will keep it simple and run Jenkins on the same machine as Eclipse. Later you will see how to set this all up to run on a machine that only had Jenkins and not the Eclipse IDE.

The following are standard Jenkins build job configuration steps, there is nothing Azure-specific about them and no special Jenkins plugins are needed other than some pre-requisites that the Azure Toolkit for Eclipse already installs on your computer. So a machine that already has the Azure Toolkit for Eclipse installed and has Ant available on the command line is ready to go. That is why the above quick experiment “just worked”. All that is remaining is to configure Jenkins:

To set up Jenkins to run Ant jobs:

1. If you have not done so already, download the jenkins.war from https://jenkins-ci.org/
2. Start Jenkins from the command line: java -jar jenkins.war
3. Use your browser to open your Jenkins instance’s home page, by default at
4. This step may look a little different depending on the version of Jenkins that you are using, but the point is to enable the Ant feature in Jenkins. In version 1.608 used for this tutorial, the following works:
   1. go to Manage Jenkins > Configure System (this shortcut may work: /configure)
   2. in the Ant section, click Add Ant
   3. Give the Ant configuration a name of your choice, for example “Ant Task”
   4. Make sure the “Install automatically” checkbox is checked
   5. Click Save

To create a Jenkins job invoking Ant to publish to Azure:

Now that you have enabled Ant in Jenkins, this is how you can configure a new Jenkins job that will build and publish your Azure deployment project:

1. In Jenkins, navigate to the create a new job page (shortcut: /newJob)
2. Set the Item Name to a name of your choice for the job, for example “Publish Hello World to Azure”
3. Select Freestyle Project
4. Click OK
5. Click the Advanced… button in the Advanced Project Options section to reveal the controls in that section
6. Enable the Use a Custom Workspace option and for the Directory enter the path to the workspace containing your Azure Deployment Project (or its parent folder, if it’s not an Eclipse workspace)
7. In the Build section, from the Add build step dropdown, select Invoke Ant
8. For the Ant Version, select the Ant configuration you created earlier (e.g. “Ant Task”), or just Default, if you don’t have any others.
9. In the Targets textbox, type “publish”. This is the target in the Azure project’s Ant build file that invokes the build and publish process
10. Click the Advanced… button, and in the advanced options, for Build File specify the relative path to the package.xml file of the project you want to build. The path is relative to the workspace directory you selected in step 6 above.
11. In the Properties textbox, you can override the properties used in package.xml with other values. The Azure Toolkit for Eclipse uses package.xml to describe the deployment package as an Ant build file. We will talk about this later, but one example of something you could override is the Azure deployment slot, by typing in, for example: “deploymentslot=production”.
12. Now click Save

Your Jenkins job configuration screen should look similar to this. The settings you need to specify are highlighted in yellow:

And that is all. To test, make sure the project is enabled in Jenkins, select it from the list of projects, and click Build Now. When it starts, open the job’s console output to see its progress to verify that it’s working properly. If it’s all set up correctly, you will see console output similar to the below screenshot:

After deployment, Jenkins should report success and provide the URL of the deployed application on Azure:

Automating Azure cloud service deployments from Jenkins on a computer without Eclipse

If you want to set this up on a machine that has Jenkins and Ant but you don’t want to install Eclipse, you first need to install a few prerequisites before you configure Jenkins, as per the below steps:

1. Make sure you’re using Java v7 or higher. Older versions of Java are not able to handle the secure communication with Azure properly and the deployment will fail. Also make sure Java is available on your command line path (e.g. on Windows, add your Java bin directory path to the PATH environment variable)
2. Copy your Azure deployment project directory from wherever you have created it using the Azure Toolkit for Eclipse onto this computer
3. Download and unzip the PackageForAzureLibrariesForJava.zip fromhttp://dl.msopentech.com/lib/PackageForWindowsAzureLibrariesForJava.html. This “all-in-one” package contains all the needed Java libraries from Azure and their external dependencies.
4. Edit the Azure project’s package.xml file to point it at the location of the Azure libraries for Java installed in step 3, by editing the location setting of the <property> element with the name “azure.lib.dir”. The XML markup in package.xml setting that property will look like this:
   

<property location="<directory of your installation of  the Azure Libraries for Java>" name="azure.lib.dir"/>

Copy your .publishsettings file from Azure into this machine and specify its path in package.xml by editing the location setting of the <property> element with the name “publishsettingspath”:

<property location="<file path to your .publishsettings file>" name="publishsettingspath"/>

This is the minimum set of steps necessary to install the prerequisites and modify your project’s build file (package.xml) to work on a machine other than the one it was created on.

There are a number of other useful settings you can manipulate in package.xml that we will discuss shortly. But for now, you are ready to set up your Jenkins job. To do that, just follow the same steps as described in the previous section, where Jenkins was running on the same machine as Eclipse, only making sure to use the proper directory paths based on where you have placed the Azure deployment project on this new machine. Specifically, remember to set the Directory of the custom workspace and theBuild File path in the Ant section of your build job in Jenkins based on where you have placed your Azure deployment project directory, then run the Jenkins job to verify that everything is working.

Updating your Java application in the Azure deployment project

If your project definition is not changing and all you need to do is update your Java application(s) in your cloud deployment, or specifically: their WAR files, editing and automation is easy. The same steps apply regardless of whether your Java application was originally referenced in your Azure deployment project by an Eclipse project reference or by a WAR file reference. In either case, what you need to do is update the resulting WAR file that the Azure deployment project is tracking before invoking the publish step of the Jenkins job.

For example, if you followed the Hello World tutorial to create your Azure deployment project a WAR file is automatically generated for it by Eclipse and placed under the approot directory of the WorkerRole1 directory. That would be the WAR file for you to update when you have newer versions of your application.

If you have added your Java applications to the project via other means, for example as direct WAR file references, then wherever those WAR files are located, those are the files to update.

How you generate that WAR file is up to you and is beyond the scope of this tutorial, as there are many different popular ways of building WAR files, using Ant, or Maven, or even Java’s simplistic jar command, for example. Or perhaps your WAR file is already built by a separate build job, in which case you can simply copy it over to the location that the Azure deployment project is pulling it from.

Whichever means of building the WAR file you use, just make sure that updating the WAR file is complete before the Azure publishing step in the Jenkins job sequence.

Modifying other deployment settings in package.xml

Modifying the publish configuration

Your Azure deployment project’s Ant build file, package.xml, contains the majority of your deployment’s description, including its startup logic, its contents and other important parameters. There is some brief documentation provided for many of those settings inside package.xml and most are beyond the scope of this tutorial. But there are a few that are very relevant to deployment automation that are worth pointing out.

At the top of the <project> element, a number of settings related to the publishing process are expressed as Ant <property>elements. They can be either modified directly in the package.xml file, or overridden in the Properties textbox of the Jenkins job’s Ant section, using the propertyname=setting syntax. Based on their name, the following is their meaning:

• azure.lib.dir – the directory of the required Azure Libraries for Java installation, which you can download fromhttp://dl.msopentech.com/lib/PackageForWindowsAzureLibrariesForJava.html.
• publishsettingspath - the file path to the required .publishsettings file you’d download from your Azure account, containing your publishing credentials
• subscriptionid – the ID of the Azure subscription you want to use for publishing your applications. This is a GUID-like string, for example: 9656ab5d-4a4a-4fe2-ae7a-4cd9fbd030ef. (Make sure your .publishsettings file includes the credentials for this subscription. If it doesn’t then it you need to get a .publishsettings file from your Azure account(s) that does)
• cloudservicename – this is the DNS prefix name that your cloud service will be published under, if you publish to the production deployment slot. For example, if your cloud service name is myhelloworldapp, then the production URL assigned to your deployment by Azure will start with that name (e.g. http://myhelloworldapp.cloudapp.net). If a cloud service by that name does not exist yet in your or anybody else’s Azure subscription, then it will be automatically allocated for you by the publish process. If it is already in use by someone else, or does not follow Azure’s service naming conventions, the publishing process will fail. If you publish to the staging deployment slot, this setting will be ignored and a random obfuscated string will be used by Azure instead. For staging deployments, you will see the URL that was generated for you in the Ant task’s output at the end of the publish process.
• region – the name of the Azure region to publish to, for example “West US”. The list of available Azure regions keeps changing, so for the latest list of available locations, check http://azure.microsoft.com/en-us/regions/.
• deploymentslot – you can set it to either Staging or Production. The Staging setting will publish your deployment under a randomly generated, obfuscated, non-discoverable DNS name. The Production setting will publish it under the cloud service name you’ve specified in cloudservicename, assuming it hasn’t been taken by anyone else yet.
• overwritepreviousdeployment – if set to true, if there is already a running deployment under that cloud service name in the same deployment slot, it will be automatically unpublished first before your current deployment is published. If set to false, such existing cloud service conflicts will end your publishing process.

Modifying the source of your WAR files

In addition to modifying settings related to the publishing process, when you are automating the deployment of your Java apps via an Azure deployment project, you may also wish to specify a different location for the publishing process to pick your WAR file from than the default role approot location used by the Azure Toolkit for Eclipse.

To change the source of your WAR file, inside your project’s package.xml file find the <component> element that hastype=”server.app”. If you have multiple applications in your project, find the one whose importas attribute matches the name of your .WAR file. When you find it, you will see that it looks somewhat like this (this assumes your WAR file is calledHelloWorld.war)

<component cloudmethod="copy" cloudsrc="auto" cloudupload="always" deploydir="%SERVER_APPS_LOCATION%" deploymethod="copy" importas="HelloWorld.war" importmethod="auto" importsrc="${basedir}\..\HelloWorld" type="server.app"/>

Let’s say you want to start pullling the WAR file for this application from a new location at /myBuilds/HelloWorld.war instead. Follow these steps:

1. The importsrc attribute should be pointing at the source of your application - in this case the source project in the Eclipse workspace. Change it to say:

importsrc=”/myBuilds/HelloWorld.war”

2. The importmethod attribute tells the Azure extension for Ant how to import the file or directory at importsrc into the role’s approot directory. The default value "auto" only has meaning within Eclipse. But we do not really need to import it into the approot at all at this time – let it get downloaded at startup from your storage account. So change it to say:

importmethod=”none”

After these simple tweaks, your <component> element will look like this:

<component cloudmethod="copy" cloudsrc="auto" cloudupload="always" deploydir="%SERVER_APPS_LOCATION%" deploymethod="copy" importas="HelloWorld.war" importmethod="none" importsrc="/myBuilds/HelloWorld.war" type="server.app"/>

From this point on, each time you republish this project, your WAR file will be picked up from the /myBuilds directory, or whatever other location you specify here.

Un-publishing the deployment

If you are just testing your application in Azure, there will be a point at which you will want to stop and delete your deployment from the cloud. This is what the unpublish script is intended for, inside the cloudTools directory. You can either run the script as is, or simply execute this command line:

ant -f ../package.xml unpublish

The unpublish parameter uses the target in the package.xml Ant build file to process un-publishing.

Conclusion

With a few simple steps you can have the publishing of your Azure deployment project fully automated and executable from the command line, from Jenkins, or from any other tool that supports running Ant build tasks. You are also now able to customize the details of your publishing configuration, including changing the source of your WAR files to be picked up by the publishing process.