Getting started with BlueMix


Recently everyone has their heads in the clouds and I decided to have a peek to find out what it is all about.

This post is a summary of my first experiences with the IBM BlueMix Cloud Computing offering and how I got started with developing my first applications for it.

Note: this is not an RTC API post. However, RTC is involved.

There are several posts of my peers. Look into Dan Toczala’s and Tekehiko Amano-san’s blog and see these posts about BlueMix:

There are more posts available.

BlueMix has been around for some time now here at IBM and I wanted to understand what it is providing. I have seen some high level presentations and demos already. Unfortunately I am not the kind of person that can learn to fly by reading books and looking at slides. I have to get things into my hands, use and experiment with it to understand how they work. This usually also involves accidents, painful crashes and recovery from them. This is however the best way for me to understand how it works and is most beneficial from my point of view.

As you can read in the BlueMix documentation about what BlueMix provides.

Citing the web site, IBM® Bluemix is an open-standards, cloud-based platform for building, managing, and running apps of all types, such as web, mobile, big data, and smart devices. It can be used to develop and run server applications.

You can use your own development environment as well as IBM Dev Ops Services to develop the applications and manage the source code.

You should familiarize yourself with the architecture of BlueMix to understand the details. I will try to use the concepts described there in the post with only a short summary what they represent.

BlueMix provides several ways to start with developing applications.

  • Runtimes are a preconfigutred set of resources used to run applications
  • Boilerplates are preconfigured containers used to run applications that usually also contain services
  • Services hosted by BlueMix provide capabilities such as session caches, persistance and other capabilities

Looking at the Runtimes there is support for Node.js applications Liberty for Java (a lean profile for WebSphere Application server), Ruby on Rails and others available.

Since I am not a Web developer but have some few JavaScript experience, I decided I wanted to go for Node.js. I got myself some material to learn about it first. After understanding the basic concepts by reading, I started to set up a local development environment.

Setup a local Node.js Development Environment

I followed http://www.nodeclipse.org/updates/ to install the local Node.js environment. I downloaded Node.js from http://www.nodejs.org/download/ and installed it as suggested in the previous link. I skipped CoffeeScript and I had a JDK 7 already on m machine.

Setup Eclipse for Node.js Development

I needed a local environment to be able to play with it and have a quick turn-around time. I downloaded Eclipse Juno, because I heard that would be the best option, and followed http://www.nodeclipse.org/updates/ to install what is needed into Eclipse.

Having done this, I was good to go and I was able to create Node.js projects in Eclipse and run and debug them locally.

I ran some examples until I felt reasonably familiar with how the language works and decided to pursue my quest to BlueMix.

Setup Eclipse with RTC

Since I intended to use Eclipse with RTC embedded to be able to use RTC against IBM Dev Ops Services, and not using Git for SCM (sorry guys, but I can’t do that), I downloaded the RTC 5.0 p2 install package from the RTC All downloads Page. After the download succeeded, I installed RTC into Eclipse. I logged into IBM Dev Ops Services from RTC using my Jazz.net ID and my IBM ID password. Weird.

However, now my local environment works with RTC and I can use any RTC repository, including IBM Dev Ops Services, to manage my work and source code.

Logging into BlueMix

I logged into BlueMix. Please note, you can use or create an IBM ID, which basically gives you an evaluation period of some months.This should be easy to follow and work like a charm.

If you are an IBM’er you can use your Intranet ID. I would suggest to do that. I unfortunately used my IBM ID and had to follow the explanation text and links to the right of the user and password fields to link both up. There seem to be still problems with this, because I happened to end up on a staging version of BlueMix that did not work for me.

Note: After logging into BlueMix, make sure your URL is https://ace.ng.bluemix.net.

Creating a Sample Project on BlueMix

To get started, I created a sample project on BlueMix.  I went into my dashboard and clicked the tile Create An App. I picked the Runtime SDK for Node.js, provided a unique host name for example rsjazz01 and accepted all the default settings.

Note: The host name needs to be unique which basically means, anyone following this will have to pick a different name and replace it in the images and text below.

The project gets opened, but won’t run, since there is nothing in there yet. In the top section to the left, underneath the application icon and name is a link named View Guide. This link provides more information about how to get started. The following is what it shows if you chose a project name RsJazzTest03. The project name will be reflected in the downloaded sample files at some places.

BlueMix Sample

Install The CloudFoundry Commandline

BlueMix uses CloudFoundry to upload and deploy applications. Follow the link and desription in the guide to download and install the CloudFoundry command-line.

Also download the example code for the application. Store the compressed code somewhere and extract the file into a Folder, for example c:\temp. Assuming the application name is rsjazz01 there would be a folder C:\temp\rsjazz01 that contains the source code of the project.

You can follow the instructions to push the example to BlueMix and run it. However, lets get it into Eclipse so that we can look at it in a more convenient way.

Create an Eclipse Project

Create an Eclipse Node.js project. It can have any name as far as I can tell, but in the context of Eclipse choose the name of the application as the project name, e.g. rsjazz01.

From the folder C:\temp\rsjazz01 that contains the uncompressed example, select all files and folders. Copy the files and folders using CTRL+c and paste them into your Eclipse project. You can do this in the Eclipse project explorer or in the filesystem. If you did it in the Filesystem, refresh the Eclipse project to see the files. The Project content should look like this:

Examlple ProjectThe main application file is represented by the file app.js. The folders public and views and their contained files are used by the framework used to create web pages.

Run the Application on the Local Development Environment

Before trying to run the application in the cloud, let’s try to run it on the development environment. In order to do so lets examine the application first. The file app.js looks as follows:

Example AppThe application prepares itself first, then gets some data, such as the host name and the port it is using from the environment, or used some defaults if not. Then it starts to listen as a server on that port and host.

The description of the sample mentions some other pieces it is using. Lets look what it is.

The files

  • manifest.yml
  • package.json

marked in the project screen shot above, are used by BlueMix to deploy and run the application. Any application that runs on BlueMix needs this kind of information to be able to deploy and run.

Lets look at the manifest.yml file first. This is the content for our sample.

ManifestThis describes some of the properties of the application, such as the host, the application name, the command to start it as Node.js application, domain, number of instances and required memory and disk. When creating an application from scratch, this is important information to look at.

The package.json file looks like this:

PackageThis file describes the application and, more importantly it describes the packages that the application requires to be able to run. It needs the Express web application framework, version 3.4.7 and the Jade Template Engine, version 1.1.4 to run on a node engine.

Install Express and Jade 

To install these packages, on your local machine, in order to be able to run the application, open a shell and use the package manager. Type each line below and hit enter. The versions needed are from the dependencies. Note, the newest version of express won’t work. There have been changes to it that will break the application.

npm install express@3.4.7
npm install jade@1.1.4

Wait for Node.js to download and install the packages.

Now right click on app.js and select to run it as a Node application. Open http://localhost:3000/ and see the web page displayed.

It is now possible to develop the application further on the local development environment. It is possible to use RTC to put it under version control, to share it and to plan the work. Any other source control providers that Eclipse supports can be used as well.

Deploy the application on BlueMix

Lets try to deploy the application on BlueMix. How this works is described in the guide above. Open a shell. The first three commands can be run anywhere.

First set the API URL for Cloud Foundry:

cf api https://api.ng.bluemix.net

Log into the server (use your own ID):

cf login -u 

This prompts for a password. Provide your password and finish the login.

Set the target space for the application. By default the space is called dev.

cf target -o  -s dev

Now change the directory to the folder that represents the project on disk, named rsjazz01 in this example. this folder is directly in the workspace folder you chose to use with Eclipse when you started it.

Now push the application to the BlueMix server:

cf push rsjazz01

The data gets uploaded, deployed and started. In the BlueMix Dashboard on the application tile you should be able to see that there are activities happening in BlueMix, while they show up in the shell. Once the process finishes the application is deployed and you can open the URL and see the same result you had from the local run.

Create a Simple Custom Sample Application

Running a sample application that is supposed to be running is – relatively – easy. But what about running a custom application? What is needed to do that?

Create a new Node.Js project and give it a name. In the example we will use rsjazz02. Pick a name that suits you if you want to perform this as well.

The new project is empty. Create a new JavaScript file and call it app.js. The file should have the following content:

/*jshint node:true*/

/**
 * New node file
 */
var http = require("http");

function onRequest(request, response){
	response.writeHead(200, {"Content-Type": "text/plain"});
	response.write("Hello World - this is rsjazz's first BlueMmix application!");
	response.end();
}

//There are many useful environment variables available in process.env.
//VCAP_APPLICATION contains useful information about a deployed application.
var appInfo = JSON.parse(process.env.VCAP_APPLICATION || "{}");
//TODO: Get application information and use it in your app.

//VCAP_SERVICES contains all the credentials of services bound to
//this application. For details of its content, please refer to
//the document or sample of each service.
var services = JSON.parse(process.env.VCAP_SERVICES || "{}");
//TODO: Get service credentials and communicate with bluemix services.

//The IP address of the Cloud Foundry DEA (Droplet Execution Agent) that hosts this application:
var host = (process.env.VCAP_APP_HOST || 'localhost');
//The port on the DEA for communication with the application:
var port = (process.env.VCAP_APP_PORT || 3000);
console.log('Start my server on port ' + port);
//Start server
http.createServer(onRequest).listen(port,host);

console.log('App started on port ' + port);

This application basically waits for an HTTP request on a port on a host and responds with a simple text. It reuses the parsing of the environment variable we saw in the sample application to get the port and the host name.

Run the application on the local Node.js and connect to it using http://localhost:3000/. It should run and provide the expected output in the browser window.

It does not have any dependencies to any other packages. However, it would not yet run on BlueMix. It lacks the information required to deploy end run it there.

Copy the manifest.yml and the package.json files from the sample application over. You can also copy the readme files, but these are not required.

Open the manifest.yml file and edit it to use a new host name. To make sure the host name is unique you can create an empty project on BlueMix, but you don’t have to. BlueMix will tell you if the host name is already taken. In the code below I use rsjazz02 as name of the application and as host name.

applications:
- disk_quota: 1024M
  host: rsjazz02
  name: rsjazz02
  command: node app.js
  path: .
  domain: mybluemix.net
  instances: 1
  memory: 128M

The line

  command: node app.js

can stay as it is. If you chose to use a different name for the main JavaScript file, you would put the name in here.

Open the package.json file and edit it to match the new situation. You can change the name and the description. Remove the dependencies, as there are no dependencies to other packages needed. Keep the rest as it is.

{
	"name": "RSJazzSampleApp",
	"version": "0.0.1",
	"description": "A sample nodejs app for Bluemix - by rsjazz",
	"dependencies": {},
	"engines": {
		"node": "0.10.26"
	},
	"repository": {}
}

Save all the changes to these files.

The application is now ready to deploy on BlueMix. Change the directory of your shell to the new folder e.g. using cd ../rsjazz02.

Now push the application to the BlueMix server using the shell command:

cf push rsjazz02

The data gets uploaded, the application deployed and you can test it using http://rsjazz02.mybluemix.net/ once it is running and the health shows green (replace the name of the application in the URL with your application). The result should be the same as in the local run.

User RTC and IBM Dev Ops Services

You can use IBM Dev Ops Services to develop and deploy BlueMix Applications with RTC. You would basically create a DevOps Services project to manage your source code and use it to deploy your application. I will try to blog about this later.

You would still do all the above steps to set up your local development environment.

Enable Eclipse To Deploy Directly to BlueMix

So far a local shell and the cf command is used to push the application up to BlueMix. As mentioned above you could also use IBM Dev Ops Services to do this.

There is a third option available. You can configure your Eclipse client to connect to BlueMix and to deploy the application automatically if you did changes, if you desire.

You can install the IBM Eclipse Tools for Bluemix into your local Eclipse Client.

Once you have done that, you can open the Eclipse View Servers and add a new server to it.

The server view would look like below. The overview shows the configured BlueMix connection. The Applications and Services shows the applications and services you have configured. The server view shows the applications on the server as well as the locally connected ones.

BlueMix Eclipse ToolsTo be able to deploy you Node.js application, you have to change it a bit first. You have to convert it to a Faceted form, using Configure in the context menu of the project.

Configure Faceted FormIn the following dialog you have to select the application type, in this case Node.js Application. Once you have done it, you can see it in the Add and Remove dialog for the server.

Configure ServerYou can add applications and remove them. If configured to do so, any save will trigger a deployment.

Summary

This post shows how you can use RTC and Eclipse to start developing Node.js applications for BlueMix. It shows how to configure the environment and the first basic steps in a way to support getting over the first questions. After reading this you should be able to do some basic experiments in half a day or so.

As always I hope this helps someone out there to save some time and I appreciate feedback.

 

Posted in BlueMix, cloud, Jazz | Tagged , , | Leave a comment

Reading and Writing Files Directly from and to an RTC SCM Stream


Can you read files directly from an RTC SCM stream or write them directly into it, avoiding having to use a repository workspace? Kevin, a colleague, was recently facing this challenge writing automation for a customer. We discussed this challenge when we met on a trip.

I was pretty sure it should be possible, but I had no clue how. I have worked with parts of the RTC SCM API and published the results in this blog. However, I have always used a repository workspace and the usual workflow and I had no answer. So how can you do this, provided the API always wants a workspace connection?

Kevin got this puzzle solved with the help of one of our developers and has published the resulting code and reasoning in this blog post. If you are interested, check his code out and give him a thumbs up!

Other posts in this blog about using the SCM API

 

 

Posted in Jazz, RTC, RTC Automation, RTC Extensibility | Tagged , , , | Leave a comment

Manage Scheduled Absences Using The PlainJava Client Libraries


I have seen questions in the Jazz.net forum around how to manage public holidays or other scheduled absences for a large user base. I have heard this kind of questions from others as well. And thought a solution would be quite interesting, so here goes.

Since I hate repetitive, boring, time consuming tasks as any one else, I wanted to do something about this for a while. In the context of this question two colleagues from Japan, Saitoh-san and Kobayashi-san approached me. They already had created a solution but were not sure how to publish it. They invited me into their IBM DevOps Services project to share what they had done. I looked into the code and found they had actually implemented an Eclipse Wizard to import scheduled absences for a user.

Since I can’t blog about things I haven’t done, I decided to take a deeper look at what they had done and create a solution from there. I finally ended up creating some tooling that allows to manage single absences and collections of scheduled absences for one or many users.

The code in this post is client API.

This blog post uses internal API which can be changed at any time.

This blog post uses internal API which can be changed at any time. If the Internal API changes, the code published here will no longer work.

Warning, some of the code uses internal API that might change in the future. If the Internal API changes, the code published here will no longer work.

The code in this post hides the RTC API for scheduled absences, which is actually an internal API, from the user. It provides methods to conveniently work with absences. It allows to create, read and delete absences for one or many users. Before we continue, the usual ceremony:

The post contains published code, so our lawyers reminded me to state that the code in this post is derived from examples from Jazz.net as well as the RTC SDK. The usage of code from that example source code is governed by this license. Therefore this code is governed by this license. I found a section relevant to source code at the and of the license. Please also remember, as stated in the disclaimer, that this code comes with the usual lack of promise or guarantee. Enjoy!

As always, please note, If you just get started with extending Rational Team Concert, or create API based automation, start reading this and the linked posts to get some guidance on how to set up your environment. Then I would suggest to read the article Extending Rational Team Concert 3.x and follow the Rational Team Concert 4.0 Extensions Workshop at least through the setup and Lab 1. This provides you with a development environment and with a lot of example code and information that is essential to get started. You should be able to use the following code in this environment and get your own extension working.

The Code

The code discussed in this post can be downloaded from here. Please note, the code might change over time, although I hope to keep the interfaces stable.

 The Absence Manager Overview

The source code of the Absence Manager is separated into three projects. The core project com.ibm.js.team.admin.automation.absence.core contains all the code required to create tooling to manage absences.

Core Absence Manager Project

Core Absence Manager Project

The package with suffix core contains the interfaces to work with, for most of the time.

  • IAbsence represents the Interface to scheduled absences in an external format used to store the data in a common format  and to make the data accessible
  • IAbsenceFactory is an interface that provides ways to create scheduled absences in the external format in different ways; the pattern to convert strings and timestamps can be set in the constructor; see the section Date, Timestamp and String Representation Troubles – Here be Dragons below
  • IAbsenceManager is an interface that the Absence Manager provides to allow to create, read and delete absences; it uses an IAbsenceFactory to create absence objects where needed

The package with suffix impl contains implementations for the interfaces that do the real work.

The package with suffix utils contains a utility class that basically manages the conversion of timestamps and string representations.

I ended up with this structure, because I wanted clear abstractions of the concepts and allow to easily enhance or replace the implementations if one so desires. Before I finally ended up with this clear structure, there where a lot of inter-dependencies in the code that where hard to handle. They tended to break the code when introducing small changes and where very confusing in general.

This is by far the most complex automation I blogged about so far and I needed to be able to test it during refactoring. Once I had the first snippets available I used a test driven approach to finalize the solution. This also made the whole refactoring required to get to a clean structure possible in the first place.

The project com.ibm.js.team.admin.automation.absence.core.tests contains unit test for the core classes and interfaces. The unit tests should cover most of the interface and its implementation. I did not make sure all is covered, but the main functionality should be covered.

  • AbsenceDataTest basically runs some simple tests to create absences in different formats
  • AbsenceManagerTest runs tests against a test repository and manages test scheduled absences in that repository for the logged in user testing, leaving the user with no scheduled absences
  • AllTests is a suite that runs all tests above

The third project com.ibm.js.team.admin.automation.absence.csv basically has two classes, that implement CSV import and -export of scheduled absences for all active (not archived) users.

CSV Absence Manager

CSV Absence Manager

The classes can be used as prototype for a custom implementation.

To read and write CSV files I used opencsv to avoid having to implement CSV reading and writing. This made it very easy to implement the functionality after the fundamental interfaces where working.

NOTE: I will not include opencsv in the download. You can download it from sourceforge, unzip it and place the library in the lib folder of the project.

There are other open and free Java implementations of CSV file readers for example SuperCSV for download as well.

  • ScheduledAbsenceCSVImporter uses a CSV file with a comma separated format to read scheduled absences and adds them to all users that are not archived
  • ScheduledAbsenceCSVExporter exports all scheduled absences for all active users to a CSV file, with a similar format, except it contains the user ID as a leading column

Here an example file for using as import source:

CSV Import Example File

CSV Import Example File

Other Uses of the AbsenceManager

If  you have a common system with an Interface to get at absence data, you can create an integration to that system with the attached code. Such an integration could, as an example, synchronize the absences between the other system and Rational Team Concert servers. The simplest approach would be to always delete all absences for a user in RTC and then recreate the absences from that system. This could be done in a nightly run.

RTC, Absences and the AbsenceManager

RTC stores the absences as java.sql.Timestamps in the CCM databse. Absences basically have the following data:

  • Summary – a text that describes the absence
  • StartDate – a Timestamp of the start date
  • EndDate – a Timestamp of the end date, the same as the start date in case of one day long absences

Absences are defined by the three attributes. To be able to find absences it is necessary to find one with the same summary and the same dates. While developing the Absence Manager, it became apparent that matching for the exact data is sometimes not desirable. Therefore the date is, in some cases, only compared to the same day, to avoid missing matches. For the summary the match is implemented as ignore-case.

It would be easy to implement a way to find all absences by the summary. This would potentially be a collection of items. For the use cases so far it was not necessary to implement it and thus I left it out.

During testing, when absences are manually created, the time created for the absence seemed to be 2pm in the timezone of the server. While specifying the absence the user actually only selects the date and not the time. If using automation, first check what the server would create and specify the times accordingly.

The RTC Absence API

The API to get absences is very easy. The code below shows how to access the scheduled absences for a contributor. All the code is hidden in the AbsenceManagerImpl.

	/**
	 * Get the internal representation of all absences of a contributor.
	 * 
	 * @param contributor
	 * @param monitor
	 * @return
	 * @throws TeamRepositoryException
	 */
	private ItemCollection<IContributorAbsence> getContributorAbsences(
			IContributorHandle contributor, IProgressMonitor monitor)
			throws TeamRepositoryException {
		final IResourcePlanningClient resourcePlanning = (IResourcePlanningClient) fTeamRepository
				.getClientLibrary(IResourcePlanningClient.class);

		IContributorInfo info = resourcePlanning.getResourcePlanningManager()
				.getContributorInfo(contributor, true, monitor);
		ItemCollection<IContributorAbsence> absences = info
				.getAbsences(contributor);
		return absences;
	}

The code basically gets the IResourcePlanningClient to get the ResourcePlanningManager and uses this to get the IContributorInfo. This contains the absences as as well as the team allocations.  The call .getAbsencs(IContributorHandle) returns an ItemCollection with all the IContributorAbsences. All the classes and interfaces, except IContributorAbsence are internal API. This is the reason why it should be encapsulated so that most of the implementation does not interfere with it. This will make it easier to adjust to changing API’s later.

The code below shows how to create a IContributorAbsence

	/**
	 * Create an internal IContributorAbsense from an IAbsence 
	 * 
	 * @param contributor
	 * @param iAbsence
	 * @return
	 */
	private IContributorAbsence createContributorAbsence(
			IContributorHandle contributor, IAbsence iAbsence) {
		ContributorAbsence absence = (ContributorAbsence) IContributorAbsence.ITEM_TYPE
				.createItem();
		absence.setContributor(contributor);
		absence.setSummary(iAbsence.getSummary());
		absence.setStartDate(iAbsence.getStartDate());
		absence.setEndDate(iAbsence.getEndDate());
		return absence;
	}

The data provided during creation is String and timestamps.

This code shows how new absences are saved using saveAbsences().

	/**
	 * Add absences from a collection to a user using the contributor object of
	 * the user. The method checks if an absence with the same summary, same
	 * start- and end- date already exist. The comparison converts the dates and
	 * uses a precision of a day to find matches.
	 * 
	 * @throws TeamRepositoryException
	 */
	@Override
	public void addAbsences(IContributorHandle contributor,
			Collection<IAbsence> absences, IProgressMonitor monitor)
			throws TeamRepositoryException {
		final IResourcePlanningClient resourcePlanning = (IResourcePlanningClient) fTeamRepository
				.getClientLibrary(IResourcePlanningClient.class);

		List<IContributorAbsence> absencesToBeCreated = new ArrayList();

		IContributorInfo info = resourcePlanning.getResourcePlanningManager()
				.getContributorInfo(contributor, true, monitor);
		/**
		 * Can't access all absences, need to narrow down to contributor
		 */
		ItemCollection<IContributorAbsence> existingAbsences = info
				.getAbsences(contributor);

		for (Iterator<IAbsence> iterator = absences.iterator(); iterator
				.hasNext();) {
			IAbsence iAbsence = (IAbsence) iterator.next();

			/**
			 * Check if the absence is already there to avoid entering it
			 * multiple times. The check is for an match of all data. It does
			 * not prevent from entering absences that overlap or different
			 * summaries. The check of the dates is not precise, but on a day
			 * level.
			 */
			if (!exists_SameDay(existingAbsences, iAbsence)) {
				IContributorAbsence absence = createContributorAbsence(
						contributor, iAbsence);
				absencesToBeCreated.add(absence);
			}
		}
		resourcePlanning.saveAbsences(absencesToBeCreated
				.toArray(new ContributorAbsence[absencesToBeCreated.size()]),
				monitor);
	}

The code for removing absences is similar, only the call is to a different method – deleteAbsences().

	/**
	 * Remove a collection of absences from the absences. Matches by summary, as
	 * well as start- and end- date. Date match is don one a same-day basis.
	 * 
	 * @param contributor
	 *            contributor to remove absences from, must not be null.
	 * @param absences
	 *            collection of absences, must not be null.
	 * @param monitor
	 * @throws TeamRepositoryException
	 */
	@Override
	public void removeAbsences(IContributorHandle contributor,
			Collection<IAbsence> absences, IProgressMonitor monitor)
			throws TeamRepositoryException {
		Assert.isNotNull(contributor);
		Assert.isNotNull(absences);
		final IResourcePlanningClient resourcePlanning = (IResourcePlanningClient) fTeamRepository
				.getClientLibrary(IResourcePlanningClient.class);

		List<IContributorAbsenceHandle> absencesToBeRemoved = new ArrayList();

		IContributorInfo info = resourcePlanning.getResourcePlanningManager()
				.getContributorInfo(contributor, true, monitor);
		/**
		 * Can't access all absences, need to narrow down to contributor
		 */
		ItemCollection<IContributorAbsence> existingAbsences = info
				.getAbsences(contributor);

		// For all absences
		for (Iterator<IAbsence> iterator = absences.iterator(); iterator
				.hasNext();) {
			IAbsence iAbsence = (IAbsence) iterator.next();

			// search if the absence is available (match same day)
			IContributorAbsence found = findContributorAbsence(
					existingAbsences, iAbsence);
			if (null != found) {
				absencesToBeRemoved.add(found);
			}
		}
		IContributorAbsenceHandle[] remove = absencesToBeRemoved
				.toArray(new IContributorAbsenceHandle[absencesToBeRemoved
						.size()]);
		resourcePlanning.deleteAbsences(remove, monitor);
	}

That’s it. Nothing big. But….

The Rest of the Code

All the 9/10th rest of the code is basically to make it easy and convenient to manage the scheduled absences and to hide the internal API within. In addition tests and usage examples make up a reasonable amount of the code.

Date, Timestamp and String Representation Troubles – Here be Dragons

The biggest trouble in the whole implementation was the conversion of date and time to timestamps. This occurs in several areas. The general problem here is that there is no general tool that is able to parse any string defining a date/time without problems.

To overcome this, the AbsenceManager uses java.text.SimpleDateFormat. This requires a string expression to parse and map the external data to the internal representation.

By default the mapping pattern used is “yyyy/MM/dd hh:mm:ss z”. However, this requires to provide date, time and timezone. If it is necessary, e.g. to avoid the time and timezone, a different mapping string can be used. To provide a different mapping string, use the second constructor of the AbsenceFactoryImpl and provide the mapping string as shown below.

		new AbsenceFactoryImpl("yyyy/MM/dd")
		IAbsenceManager absenceManager= new AbsenceManagerImpl(teamRepository, new AbsenceFactoryImpl("yyyy/MM/dd"));

Please note, if no time is provided, the server will pick an hour on its own.

The IAbsenceFactory provides also ways to create new absence instances with different representations. However, keep in mind that some comparisons are done internally and it is best to have a common schema.

How to use the AbsenceManager

How to use the absence manager is shown in the classes AbsenceManagerTest and ScheduledAbsenceCSVImporter.  You have to be connected to a team repository with a user that has sufficient permissions to read/write the absences.

To get the AbsenceManager use:

		IAbsenceManager manager = new AbsenceManagerImpl(fTeamRepository,new AbsenceFactoryImpl());

		String absence1Summary="Absence 1";
		Date today= new Date();
		IAbsence absence1= manager.getAbsenceFactory().newInstance(absence1Summary, today);
		IAbsence absence1_sameday= manager.getAbsenceFactory().newInstance(absence1Summary, new Timestamp(today.getTime()+60000));
		IAbsence absence2= manager.getAbsenceFactory().newInstance(absence1Summary, "2014/07/17 02:00:00 CEST");
		IAbsence absence3= manager.getAbsenceFactory().newInstance(absence1Summary, "2014/07/17 02:00:00 CEST", "2014/07/30 02:00:00 CEST");

You can add single or multiple absences like this:

		// Add a single absence using a contributorHandle
		manager.addAbsence(contributorHandle, absence1, monitor);
		// Add a collection of absences
		ArrayList<IAbsence> addAbsences= new ArrayList<IAbsence>();
		addAbsences.add(absence1);
		addAbsences.add(absence2);
		manager.addAbsences(contributorHandle, addAbsences, monitor);

		// Add a single absence using a userId
		manager.addAbsence(manager.getContributor("ralph"), absence1, monitor);
		// Add a collection of absences
		ArrayList<IAbsence>addAbsences= new ArrayList<IAbsence>();
		addAbsences.add(absence1);
		addAbsences.add(absence2);
		manager.addeAbsences(manager.getContributor("ralph"), addAbsences, monitor);

The interface implements a convenience method getContributor(String userID) to allow to get the IContributor interface, which also implements IContributorHandle, from the userId.

You can get absences for a user.

		ArrayList<IAbsence> userAbsences=manager.getAbsences(contributorHandle, monitor);

You can test for an existing absence

		if(manager.hasAbsences(contributorHandle, absence1, monitor)){

You can remove specific absences

		manager.removeAbsence(contributorHandle, absence3, monitor);
		manager.removeAbsence(contributorHandle, absenceCollection, monitor);

You can delete all absences up to a specific date for

		// Clear all absences for user
		manager.purgeAbsences(contributorHandle, null , monitor);
		// Clear all absences up to a certain date for user
		manager.purgeAbsences(new Date(), monitor);
		// Clear all absences for all users up to a certain date (including archived users)
		manager.purgeAbsences(new Date(), monitor);
		// Clear all absences for all users (including archived users)
		manager.purgeAbsences(null, monitor);

Import the code

The code can be downloaded here. Save the code on the local disk. Set up an Eclipse client for example the RTC Eclipse client. Follow the section Setting Up The Plain Java Client Libraries and Setting Up Your Workspace for Plain Java Development in the post Setting up Rational Team Concert for API Development to set up at least the Plain Java Client Libraries. You don’t have to set up the SDK to run the code. This step would however provide you with access to the API classes and their source code. Make sure to create the user Library for the Plain Java Client Libraries.

Import the compressed file as archived project file into Eclipse.

The example does not ship the opencsv library. Download it by following the link e opencsv  and clicking on the download link in the General section of the description shown below.

Open CSV download link

Open CSV download link

Store the download file e.g. opencsv-2.3-src-with-libs.tar.gz in a temporary folder. Use 7Zip to extract the file. Use 7Zip again to extract the file content. Find the folder deploy in the extracted folder structure. E.g. C:\temp\opencsv-2.3\deploy  and copy the enclosed JAR file e.g. named opencsv-2.3.jar into the folder lib underneath the project com.ibm.js.team.admin.automation.absence.csv. Please note, opencsv also ships the file junit.jar. that is not the file you want.

For all the projects you just imported starting with com.ibm.js.team.admin.automation.absence.core. Run a clean build and check the build path for errors.  If you used the proposed name PlainJavaApi, for the Plain Java Client Libraries user library you should be fine. If you see errors, you probably have a different name. In this case configure the build path. Remove the Plain Java Client Library user library from the build path and add your user library.

Finally all errors should be gone.

Run the ScheduledAbsence Code

You are now ready to use the code and run the Unit Tests, the CSV importer and the CSV exporter.

The code ships with launches. They are located in a sub folder named Launches in the projects.The launches should now be available in the Eclipse Debug Configurations and Run Configurations menu.

Open the configuration e.g. for the ScheduledAbsenceCSVImporter launch. The Arguments tab will show

"https://clm.example.com:9443/ccm/" "ralph" "ralph" "USFederalHolidays2014.csv"

The  ScheduledAbsenceCSVImporter  and the ScheduledAbsenceCSVExporter require

  • the Repository URL
  • a user ID
  • a password
  • a name for the CommaSeperatedFile

Replace the information with your own values. Then you can run the importer and the exporter.

The same information is required in the AbsenceManagerTest Junit test. It is hard coded in the AbsenceManagerTest. Replace the values with your own information if you intent to run the test against you own test system.

Next Steps

I will try to find some time to be able to create a version that can be shipped as binaries with batches so that they can easily be run from the command line. In general the post Understanding and Using the RTC Java Client API should give you all the information you need to get this done yourself.

Summary

This post provides you with all the code and information needed to automate managing scheduled and other absences in RTC repositories. The code is obviously not production code, so you should make sure it works as advertized in your environment. the provided tests should help you to fix errors, should they occur.

As always, I hope this post saves users out there some time – or is at least fun to read.

Posted in Jazz, RTC, RTC Automation | Tagged , , , , , , , , | 1 Comment

New Version – Does Your Backup Still Work?


Just upgraded? Does your backup really still work?

Upgrading to new versions of the Jazz based Collaborative Lifecycle Management solution is important. However, the upgrade does not necessarily stop with the upgrade process. It is also very important to check if the backup procedure still works and covers all required data.

Since RTC 1.0 was released, the Jazz based solutions have undergone several changes in the storage architecture, adding new applications, databases and index files that require back up.

Adjust and check the backup procedure for version 5.x.

Version 5.0 is another case where this happens. The Requirement Management application Rational Doors Next Generation now gets its own database storage and index files. Before version 5.o the JTS was used to store and index the RM data. It is important to add the database and the index file location to your backup.

In the Backup CLM Deployment Wiki page we try to explain the backup steps for your solution. The page has been updated for the version 5.0. Please carefully check if your upgrade still works. It is also a good idea to try a restore on a test system, to make sure your data is valid and can be used.

Posted in CLM, Jazz | Tagged , , , | Leave a comment

Is The Extension Deployed? How Can I Redeploy?


This is a question that comes up every so often in the forums. Unfortunately there is no place, I am aware of, where this really gets explained and it seems to require the knowledge of mysterious URL’s. Is there an easy way?

I wanted to always blog this, but somehow I just never did. Let’s unveil it now.

Internal Tools

The best way to check if a server extension is deployed on a server is to use some Internal Tools. These hidden Internal Tools are also the easiest way to enforce redeployment of the applications and added extensions.

The internal tools can be accessed by injecting the string ?internal=true into the server administration URL for a Jazz application. This makes it available on the Server Administration page. It is possible to append the string above behind the URL for the main administration pages and then open the server administration. If the server administration page is already open and it contains an action the string needs to be injected before the # (hash tag) separating the action from the base URL.

The image below shows the injected string in the server administration page before the action.

InternalTool_Injected

The image below shows the URL when just appending the string before opening a specific server administration page.

InternalTool_1An URL like

 https://<server>:<port>/jazz/admin?internal=true#action=jazz.viewPage&id=com.ibm.team.repository.server

shows the internal tools menu that can then be used to look at data that is usually not revealed.

Use the Component Status menu action to check if an extension is deployed. It opens the Component Status page with all components that are deployed. You can then use the browsers find functionality to search for your extension, or rather the name of the component you chose. If you followed my advice and chose a unique and easy naming schema, you should be able to find it. The image below shows the example of the RTC Extensions Workshop at the end of the list. Searching for rtcext revealed this in no time.

DeployedComponent

If your component does not show up, fix your deployment and try again.

Server Patch Extension

** Update ** Eric suggests another way to be able to see your extension in the comments below. Use the Server Patch Extension – https://jazz.net/wiki/bin/view/Main/ServerPatchExtension so that your plugin is listed on the CCM Admin Page

Force Redeploying using Internal Tools

The runtime of the Jazz servers caches information about deployed applications. If a new version of an application is deployed, the server would not pick that up and would not redeploy it. To enforce redeploying, it is necessary to request a server reset. There are several ways to do that.

One way is to use the Internal Tools and click on the Server Reset menu action. On the page displayed, click the Request Server Reset button. Next time the server is started, all plugins are redeployed.

Force Redeploying – Alternative Approach

If your server is not up, another way to enforce a server reset is to search for the file built-on.txt in the work folder of your application server. If this file is deleted, a server reset is performed the next time the server starts. For Tomcat you can find the file in the work folder of the application.

Built-on

Summary

The Internal Tools provide an easy way to find out if a custom component is deployed and to request a server reset.

As always I hope this helps practitioners out there to be more effective.

Posted in Jazz, RTC, RTC Automation, RTC Extensibility | Tagged , , | 4 Comments

Running The RTC 4.x Extensions Workshop With RTC 5.0


Since the new 5.0 version of Rational Team Concert and the CLM tools is out now, would the Rational Team Concert 4.x Extensions Workshop still work with this version?

The short answer is: Yes!

I just quickly tested if the Rational Team Concert 4.x Extensions Workshop works with the newest release 5.0 and I was able to successfully smoke test it.

To test if it still works, I performed Lab 1 completely. The new setup tool introduced recently ran like a charm and this was successful.

I then ran Lab 2 and finally Lab 5 with the complete code accepted. All labs worked as advertised. I would not expect any surprises in Lab 6 (except the normal issues when trying to deploy).

Observations

As there are some things that I notice that are different from the 4.x versions. I will summarize them here.

After setting up the SDK an error shows up in some of the project explorers. The error looks like below in the project explorer:

ExternalPluginError_1It seems to be an issue with the build path if one looks at the details:

ExternalPluginError_2I can’t remember having seen this in the past. Currently I have no solution to get rid of it either. However, it only seems to come into the way when launching and does not seem to have any ill effects.

To get around it, when launching, use this dialog to skip this issue:

ExternalPluginError_SkipIf you check the ‘Always launch without asking’ option, be a ware that this could be problematic if your own code has errors as well.

In the other labs, the only thing that seemed to be different is that the Eclipse password secure storage is getting more persistent. You should probably provide a password, to avoid having to deal with it every time.

Summary

So you can run the Rational Team Concert 4.x Extensions Workshop with the current Rational Team concert version 5.0 and it is likely it will run also with later versions.

As always, I hope the code above helps someone out there with extending RTC.

 

Posted in Jazz, RTC, RTC Extensibility | Tagged , , | Leave a comment

Get an Item From its URL


This came up on the form recently. It is interesting, because in several areas, including the web UI the URL is what is exposed and not an item ID.

I basically browsed the RTC SDK (using my standard setup described here) and searched for java.net.URI as parameter.

I finally found com.ibm.team.repository.common.Location being used for this purpose. You can basically create a location from an URI and get to the object. There are several methods to get at the item handle. For example Location.itemOidUriToHandle(). See Arne’s answer to his own question for details how he does the trick in an example. To be able to find it in searches, here the code:

URI snapUri= URI.create(snapshotIdentifier.replaceAll(" ", "%20"));
           
Location snapLoc = Location.location(snapUri);
IBaselineSet snapshotItem = (IBaselineSet)
         teamRepository.itemManager().fetchCompleteItem(snapLoc,IItemManager.DEFAULT, monitor); 

I used Location also in this post to create links with back links between items.

Posted in Jazz, RTC, RTC Automation, RTC Extensibility | Tagged , , | Leave a comment