Setting up Rational Team Concert for API Development

There seem to be still a lot of questions around how to develop against the RTC API. I will try to cover the best practices I developed over the years in some posts on this Blog over the next few months. I will try to cover at least the following topics over time:

• What you have to expect (this post)
• Setting up your own development environment to develop against the SDK and the Plain Java Client Libraries (this post)
  • Also see the Workshop Overhaul post, the RTC 5.0 workshop post and for the 6.0.3 version the post Updated RTC Extensions Workshop for RTC 6.0.3. Look here for additional posts for newer versions of RTC.
• Understand what API’s are available
  
• Understand your development environment and how you can use it to develop client  Extensions and Plain Java Client Libraries based applications
  
• Understand how to use the capabilities of the development environment to find the API you are interested in and how it works (See this post and this post for details)
• Understand how you can debug Extensions and Plain Java Client Library based code
  
• Understand how to clone your development workspaces and how to backup your test data
• Understand the differences and similarities of the client and the server API
  
• Understand the impact of newer versions of the tools on your extension or automation work
• Find and explore existing Extensions Points
  
• See this Wiki Entry for extension points and operation ID’s;
• Find ID’s of elements for example to be able to extend context menus or hide context or other menu entries
  
• The Interesting Links page is maintained for more hints
• The RTC Extension Workshop Overhaul
  
• Solutions for issues with the Extensions workshop for Version 5.x: Running The RTC 4.x Extensions Workshop With RTC 5.0.x
  
• Running the RTC Extensions Workshop With RTC 6.0
  
• Running the RTC Extensions Workshop With RTC 6.0.1
  
• Running the RTC Extensions Workshop With RTC 6.0.3
  

This post will provide some guidance on how to set up your development environment for SDK and Plain Java development. But before we look into that, some information on the general challenges involved with doing this kind of development.

If you want to skip the valuable introduction that follows, skip to “Getting Started”.

Preface

“All hope abandon ye who enter here”

Well, it is not that bad, really. Developing automation and extensions can be loads of fun and a very rewarding task. However, I have learned the hard way, that it can also be very challenging and I would like to use this section to set, or correct, some expectations and make some general statements about the domain you are going to enter, should you decide to continue your quest.

The SDK and the Plain Java Client Libraries are an Eclipse Framework based Java API. To work against it, will require Java skills. That should not stop you from trying, if you are only a Java beginner, but you should be aware that you might have to dig into how to do certain things in Java, which will cost time.

Because RTC and Jazz is based on Eclipse, even on the server, you will be exposed to a lot of the Eclipse extensibility and programming model. A lot of advanced frameworks and APIs are used underneath the API you will use. These frameworks, EMF as an example, are very performant and versatile. On the other hand they are also using advanced programming patterns and models. Therefore they are often very abstract and it is sometimes hard to understand the relationships of objects.

Especially EMF is far more abstract as PoJo’s. EMF provides capabilities to generate code for domain models from abstract models such as UML. The input model allows to describe objects, their attributes and the relationship between these objects. The code generated is very efficient and clearly separates model and implementation. It uses generated interfaces to represent model objects and generated classes that represent the implementation of the objects. Using Interfaces allows implementing multiple interfaces and multiple inheritance. This makes it sometimes hard to understand the structure and the purpose of objects, how they relate and how to get from one to another. Knowledge about EMF is not required, but will help to deal with finding around in the API.

In addition to using EMF, Jazz/RTC also uses a database to store its information. This means you have to look up data using specific API. To increase performance it does also not always provide you with all the related data, but with handles that you have to resolve. One example is work item attributes. A work item needs to be found and resolved from a handle. It provides you with only very few methods to access its basic data such as its ID. If you want to access more attributes, you have to look up the attribute interface for the specific attribute ID first and then use the information found to access the value on the work item. This means you will have to find and understand a process with several steps to get at the data and the services needed to get it.

If you intent to create client or server extensions, you should have at least a basic understanding of the how Eclipse Extension Points and Extensions work. It is possible to to do this without any knowledge, but it is not easy. If you don’t have experience with Eclipse extensions, I would suggest to get a book about how that works and how to develop them. There are several on the market and they will make things easier for you over time. Eclipse extensions use a declarative approach as well as Java code. Without any background it is easy to get into a situation where things don’t work and the reason is just a mystery. This is especially true when working with the plug-in XML and during deployment of plug ins.

RTC covers a lot of ground. It supports managing Work Items, planning with them, provides SCM, Build and process enactment. The process component allows modeling projects, teams, timelines, roles and ties the other capabilities together. This means a lot of capabilities and a lot of API. Even if there are areas where there is no official API yet, like for planning.

If you get started, it will feel basically like being stuck in your village 1000 or more years ago, with no map and the task to explore and conquer the world. At least that is how it felt (and still feels) to me. So the best you can do in the beginning, is to look at your task and find some information on how to get started.

Other than 1000 years before, nowadays we have a pretty global village and ways to get at information that were not available even half a century ago. So you can search the internet for examples that have an overlap with what you are up to. You can ask in the Forum and build up your network. If you ask in the Forum make sure to tag your question with the tag extending. If you are lucky and find information, you will have to find the missing pieces and discover new areas. Sometimes you will have to cross seas or chasms to explore what lies beyond. Over time you will create yourself a map of the world that is more or less detailed and more or less accurate. It will most likely have white areas and places described with “Here be Dragons”. So prepare yourself for exploring new areas every time you want to do something different. Just be aware you probably haven’t been everywhere yet and you might not know the customs of the other areas. Exploration is hard and sometimes dangerous. Keep reminding you of this fact. Be prepared for a rough ride with great rewarding discoveries on the way.

If there would be a complete map of this world, I would be happy to share it. As it is, the best I can do is share what I, and others, have explored. I and your other fellow explorers would be really happy if you share information on areas you discovered to make it easier for the ones that follow in our steps. This blog provides you with examples and links to blogs of other explorers in this realm that are sharing pieces of the map discovered so far. I will try to maintain a link page with pieces of the map other explorers have found.

Getting Started

The natural starting point is how to set up your environment for development. Being able to set up a development environment for the RTC SDK and the Plain Java Client Libraries still seems to be a hurdle when starting developing against the RTC API. So lets have a look how that works. For the books, I always wanted to share this information, but there were always more pressing matters. My apologies.

Setting up the RTC SDK

Regardless if you want to develop client or server extensions or if you just want to work against the Plain Java Client Libraries, from my experience you should first set up a RTC SDK based development environment. As you will see later on, what you will gain is worth the investment.

Follow the Rational Team Concert Extensions Workshop Lab 1 (See the beginning of the post for newer versions) as described below, to perform this task. I don’t intent to rewrite the workshop. This section will only try to provide you with some more context and additional information that makes it easier to understand which steps are important and which have to be done (or can be left out) to set up the development environment.

If you intent to develop Eclipse extensions for the Eclipse Client or the RTC Server, you should run this RTC Extensions workshop at least once. It provides you with a step-by-step description you can follow. It also provides you with useful example code that touches some areas of the RTC API.

* Update * there was a major overhaul of the RTC Extensions Workshop. The text below has been updated to reflect the new workshop.

* Update * there are some problems with the SDK when Running The RTC 4.x Extensions Workshop With RTC 5.0.x. Check the post for how to solve them or use 4.x for development – the APIs are mostly unchanged.

* Update * See also Running the RTC Extensions Workshop With RTC 6.0

When can I skip doing the Extensions workshop?

As described below, there are cases where you may be able to skip all or part of the Extensions workshop in order to only setup the development environment. When can you skip doing the workshop and when should you do it?

If you want to develop extensions for the server, you can skip the workshop if

• You have done the workshop already
• You are a Jazz developer that develops with the server API every day
• You are are a subject matter expert that has already successfully developed a bunch of custom follow up actions and successfully deployed them.

If you only want to code against the Plain Java Client Libraries you can also only do Lab 1 and skip the rest of the workshop.

In all other cases you should definitely take the time to do the RTC Extensions workshop.

Minimal Setup Steps to Setup Your Development Environment

If you only want to code against the Plain Java Client Libraries, or if you want to just set up the SDK, the part of the workshop you should perform is Lab 1. At least run through the sections 1.1 Download and Unzip the Required Files, 1.2 Setup for Development. If you just want to do Plain Java Client Libraries you can skip all the following labs. You can basically scroll down to the section Setting Up The Plain Java Client Libraries below.

If you want to setup only the SDK you don’t have to run 1.3 Setup the RTC Tomcat Server,  to setup the development RTC server and you don’t have to run the WorkshopSetup tool. You don’t have to connect the Eclipse client to the Tomcat RTC workshop server at https://localhost:9443/ccm/ if you skip 1.3, instead you would likely connect to your team’s SCM system to version your code.

When setting up the SDK 1.4 Complete Setup of Your RTC Eclipse Client is only necessary if you want to use the SDK to develop client and server extensions. In this case I strongly recommend to run all the labs. If you want to test the extensions on a Tomcat based server you have to at least  run 1.3_12 in 1.3 Setup the RTC Tomcat Server, to be able to connect to that server.

Please follow the advice in the workshop for setting up the Eclipse workspace and use a folder named /workspaces/Dev1/WS . A second workspace would be named /workspaces/Dev2/WS . This greatly simplifies workspace management later.

You should use different folders to develop for different Versions of RTC. I usually follow the pattern /RTCDev, where version is the version of the release. For example for 4.0.1 /RTC401Dev/ as root. see the image blow. It is a best practice to keep the file name short, especially on Windows, because that OS struggles with long path names you tend to get with Eclipse.

If you want to develop client or server extensions, I strongly recommend to set up a test server based on Tomcat and Derby for deployment tests. I use the JKE Banking aka Money that Matters example for all my initial tests. If necessary, I deploy custom templates and create customized project areas on my test server. It is possible to backup the test server to be able to repeat deployment tests without always having to set up a new test server. With Tomcat and Derby you can zip the complete install folder for backup and unzip it to restore. This can be improved if you just backup and zip the data required. You can find some more details about what is required for backing up here.

As a general best practice, at least on Windows, use 7Zip to zip and unzip files related to Jazz and RTC. The zip tool built into Windows has problems with long path names and just does not work reliable. See the article Workaround: Rational Team Concert SDK .zip fails to extract on Windows because of long file path for details.

To run your own Tomcat test server for debugging, you have to modify the server.startup.bat file of this test server as described in the beginning of 1.3 Setup the Tomcat Server. If you do, run the rest of Lab 1 and make sure you can do the basic debugging steps as described in the last sections of the lab with your own test server.

If you run the test server on your development machine, make sure to avoid using conflicting ports as described in the RTC Extensions Workshop.

In my experience it is not necessary to run the workshop on the exact version you finally want to develop for. You can run the workshop on any supported version. Later you can set up the SDK for the version you want to use, as described here and bring the source code over. As an alternative, if there are issues with the versions (which should no longer be the case) you can set up the workshop on a supported version, upgrade the repository to the version you want and run it. Because the code did not change since 3.x, I don’t think it is worth the effort.

Running the Workshop – Additional Information

Please Note: If you run the workshop, make sure to follow it closely for the parts you are doing. This workshop has been run many times in classes. The experience is, it works really well if you pay attention to the details. This holds especially true for Lab 1. If you miss to check a check-box, or push the wrong button, or import the wrong file the wrong way with wrong selections, you will have issues. In a case like that, the only chance you have is to delete the wrong import, or whatever it was, and do the step again. The trouble is to understand the issue from the symptoms later, if you don’t have the experience. In this case you might have to start all over, delete what you did so far and repeat all steps.

You should be aware that the workshop uses two different RTC/CLM servers. The server you set up in 1.3 Set Up The RTC Tomcat Server https://localhost:9443/ccm/ as displayed below, is an external workshop SCM and Test repository simulating a production environment. It is used to get the source code of the workshop example. The code is stored in the repository in several states of development each marked by a baseline. In addition this workshop server is used as deployment target in the final lab.

The second server that you use in the workshop is the Jetty based Test Server you run in debug mode with the public URI https://localhost:7443/jazz/.

It is a best practice to use a Jetty based test server to develop and test your code as described in the workshop, before attempting to deploy on a test system and finally in production. The reason is simple, it is much faster to develop this way. You can get your extension stable and you will be confident it works. In the final deployment tests on Tomcat or WAS you will likely only have to deal with deployment problems.

In a real production environment, you would connect to your development repository RTC server and use its SCM system to store your code. You would have your own test RTC server, as suggested above, that you use for your final deployment tests. This test server needs to be enabled for debugging, if you want to use remote debugging as mentioned above. In the workshop the Tomcat based RTC server is used for both purposes, like in a simulation.

Working With Your Development Workspace

If you followed the steps above, you have now a working Eclipse workspace as a development environment set up with the SDK. If you develop Extensions you can use this workspace to debug your extension, search for code in the SDK and do all kinds of cool things. Some of these capabilities are used in the Extensions Workshop. The key capability is, the SDK provides you with source code for many of the API classes which allows searching for the API.

It is a best practice to create a copy of your Eclipse workspace with the SDK set up and all its settings by just duplicating /workspaces/Dev1/ and naming the duplicate for example /workspaces/Dev2/. This duplicates all settings, including launches and jetty test servers. You can also use 7Zip to zip the folder /workspaces/Dev1/ and all its content as a backup.

You can also share the projects and files you imported into your workspace in the Jazz SCM. I would suggest to use a dedicated component. Please be aware that you still have to setup the SDK target platform and have to hook up the SDK source code if you start with a fresh workspace. Again, you can copy or zip/unzip your workspace that has been set up to sped up things and then load the component with the imported files.

Launches and Jetty Server Configuration

If you have run the Rational Team Concert Extensions Workshop and want to use the Feature Based Launches for a different RTC Version or Workspace, you can copy them over into the desired location. You can find these files in the Eclipse project RTC Extension Workshop Configuration.

Use File>Import, select Existing Projects into Workspace. Then browse to the workspace that contains the project. Select the project to import and check the check box for the Copy projects into workspace option.

You can rename or remove launches for the client and the server and organize them as needed.

Make sure to remove the XML files in the conf folder and replace them with the ones for your current version.

Development, Debugging, Deployment of Server Extensions

You should use the techniques described in the Rational Team Concert Extensions Workshop to develop, debug and deploy your extensions.

Always use Jetty to test if server extensions solution can be deployed and work there first. After that is proven, you can be reasonably sure that it can be deployed on a real application server.

You can only be reasonably sure, because your extensions dependencies might still contain dependencies to code that is not available where you deploy it, or might experience problems with the class loaders of the application server.

However, your extension will then likely deploy and, in case there are unsatisfied dependencies or other issues, at least create an error message.

Use https://localhost:9443/ccm/admin?internal=true to see the internal tools that can help to request a server reset, see the provisioning status and the component status of your extension.

You can use

• Server Reset to enforce a new deployment of all extensions with an URL like https://localhost:9443/ccm/admin?internal=true#action=com.ibm.team.repository.admin.serverReset
  
• The Provision Status to see the provisioned update sites using an URL like https://localhost:9443/ccm/admin?internal=true#action=com.ibm.team.repository.admin.provisionStatus searching the page for your ID’s
• The Component Status to see if your component has been deployed successfully enough to be shown using an URL like https://localhost:9443/ccm/admin?internal=true#action=com.ibm.team.repository.admin.componentStatus
  

If you deploy and the deployment fails catastrophically, you can see this in the server logs e.g. as exception. If you fail to deploy correctly and, as an example, the ini file for the update site points to a non existent folder, you server might be unable to start and the links above won’t work.

Debugging Enterprise Extensions with Jetty

The Enterprise Extensions are not shipped with the SDK. If you need them for debugging in Jetty follow this answer on Jazz.net.

Setting Up The Plain Java Client Libraries

If you want to develop client extensions or Plain Java Client Library based automation it is recommended to set up the Eclipse workspace to use the Plain Java Client Libraries. You can use most of the code developed with Plain Java Client Libraries also for Eclipse client extensions and debugging of Plain Java Client Library code is faster and easier than debugging in a dedicated debug Eclipse client.

If you followed the Rational Team Concert Extensions Workshop Lab 1 you have already downloaded and installed the Plain Java Client Libraries.

If you haven’t, here is what is needed to download and extract them. Open the All Downloads tab of the RTC version you are interested in. For example https://jazz.net/downloads/rational-team-concert/releases/4.0.1?p=allDownloads and scroll down to the Plain .zip Files section. Look for the Plain Java Client Libraries downloads. There are the libraries as well as the JavaDoc.

Download both files into a temporary folder.

Use 7Zip as explained in the workshop and unzip the Plain Java client Libraries download file (for example named RTC-Client-plainJavaLib-4.0.1.zip) using 7Zips Extract Files and provide the extraction Path /Installs/PlainJavaAPI .

Now do the same with the JavaDoc for the Plain Java Client Libraries e.g. a file named RTC-Client-plainJavaLib-API-javadoc-4.0.1.zip. As extraction path provide /Installs/PlainJavaAPI/ as path again.

You should end up with this directory structure.

Now you have installed the Plain Java Client Libraries. Familiarize yourself with the content of the folder PlainJavaApi. It contains the Jar files you need if you run Java code that uses the Plain Java Client Libraries. The doc folder contains the JavaDoc that you can open using the index.html file in that folder. The license folder contains the license files for it.

The snippets folder contains example code and a README.TXT file with instructions how to run the snippets. Basically you have to provide reasonable defaults for the parameters, compile the snippet code and run the snippets against a test repository. The snippets require the parameters repositoryAddress and snippetUserAndPassword. You can either set them as environment variables, or you can change the parameter default in the source code of the snippets displayed below.

You can edit the source with any editor, compile it with a JDK and run it from a shell. However, it is hard to debug this and it is impossible to search the API that way. Lets change that.

Setting Up Your Workspace for Plain Java Development

It would be great if we could run this in Eclipse and look for the API classes, similar to creating Eclipse extensions. The next sections show how that can be done, if you have setup your Eclipse with the SDK as described in the Extensions Workshop as described above.

Start your Eclipse client with the workspace set up for the SDK. You should see a few projects already that appeared during the setup for the SDK. Now we want the Plain Java snipptes to be usable in Eclipse for debugging.

Create a new Project using File>New>Project. In the wizard type Java in the filter or select Java>Java Project. Click Next. Name the Project PlainJavaSnippets and click Finish. Let Eclipse change to the Java Perspective. Now unfold the project, click on the src folder and create a new package. Name it snippets. Right click on the new package and select Import in the context menu. In the wizard select General>File System.

Click Next. Browse for /Installs/PlainJavaAPI/snippets. Select the folder snippets. In the Into Folder selection field, make sure it shows your package src/snippets in your PlainJavaSnippets project as presented below, then click Finish.

Now you have the code imported in your project and it shows errors, rats! But that is just because it is missing all the classes from the Plain Java client Libraries.

So lets fix the errors first. right-click on the project and select Build Path>Configure Build Path.

Open the Libraries tab in the properties editor, then click Add Library. In the Add Library dialog select User Library and click Next. There is no User Library yet. Click on the button User Libraries… On the next dialog click New.

Name the Library PlainJavaApi and click OK.

On the next dialog click Add JARs… In the following dialog browse to /Installs/PlainJavaAPI/. Select all JAR files in the folder and click Open.

Your library should show a lot of JAR’s now. Click OK.

On the next dialog make sure your new User Library PlainJavaApi is selected and press Finish. The Library should now show up in the build path. Click OK to close the Build Path Editor.

All the compiler errors should be gone after a moment. You can now run and debug your Plain Java Client Library snippets in Eclipse. Next time you want to create a new project, you can just re-use the library you have just created.

If you start new Plain Java Client Library based projects, just add this library to the build path and you should be ready to go.

Open Snippet1.java. and look into the main method. right click at TeamRepositoryException and select Open Declaration.

You will see a Class File Editor and, unless you installed a Java Decompiler, no code. Dang, we had hoped for more. And, we can actually achieve more, just using a little dirty trick described in the next section.

Make the SDK Source Code Available for the Plain Java Clients Libraries

This only works if you have installed the RTC SDK in your Eclipse following the Extensions Workshop. If you did not and just wanted to shortcut, this a bit, consider following the original advice.

The reason why we can see the code in plug-in development for the SDK is the Eclipse PDE – the Plugin Development Environment. Unfortunately that works only for plug-ins. We don’t have a Plugin – yet. We will have one in a minute and trick the PDE into showing us what we want.

Right click on the PlainJavaSnippets project and select Configure>Convert to Plug-in Projects…

In the wizard make sure your project is selected and click Finish.

Your project is now a plug in project. It will still not show any code. You have to tell it where to look first.

Unfold the new folder META-INF and double click at the file MANIFEST.MF.

A plug in editor opens. Open the Dependencies tab of the editor and click Add…

In the Plugin Selection enter org.eclipse.core.runtime and add the dependency by pressing OK.

The same way enter

• com.ibm.team.repository.client
• com.ibm.team.repository.common

Basically look into the import section of your snippet and add every dependency using the package name.

Save the changes to the plug in manifest.

Now we should have everything done to get the source code. However, the order of things to look up might cause an issue. Open the Configure Build Path dialog again as described above. Open the Order and Export tab and make sure the PlainJavaApi library is below the Plug-in Dependencies entry. Use the Up and Down button to achieve this, then click OK.

Now open your snippet again and use Open Declaration on TeamRepositoryExceptioen in the main method, as you did above. You should now be able to see the code that was imported in the SDK.This works, because the SDK and the Plain Java Client Libraries are basically the same code, but packaged in a different way.

This makes life a lot easier, because you can now use debugging and search and all other means available in the PDE to find the API you need. We will explore what we can do in future posts. See Understanding and Using the RTC Java Client API for more details on the client libraries.

Please Note: You can add more dependencies to the manifest, one for each package you need to look at. You should just make sure it is client API that is available in the Plain Java Client Libraries. You should be able to use Eclipse code completion with CTRL-Space for all the classes in the Plain Java Client Libraries. If you find a class and don’t see the code, try to add the package to the manifest. Please be aware that the SDK does not package the complete code. There are classes that don’t provide the source code.

It is a best practice to create a copy of your Eclipse workspace with the SDK and the Plain Java Client Libraries set up and all its settings by just duplicating /workspaces/Dev1/ and naming the duplicate for example /workspaces/Dev2/. This duplicates all settings, including launches and jetty test servers. You can also use 7Zip to zip the folder /workspaces/Dev1/ and all its content as a backup. This can save you a lot of time if you want to set up a fresh workspace later.

Summary

Setting up the SDK and development for the Plain Java Client Libraries helps you to explore and use the unknown lands of the RTC API and enables you to develop much more efficient than before. You should always invest the little effort to set up your environment this way.

I hope sharing this will help users out there to understand and develop the RTC API more efficient and provide more automation and extensions in less time.