RTC Extensions Workshop – How to fix SSL protocol errors preventing connection to Jetty Debug server

Since some time now I started to run into a blocking issue with the Extensions workshop. I was not able to find a solution so far. Today a colleague saw the same and asked for help. Here what I found.

Problem: the browser prevents connecting to the Jetty server

When launching the Jetty Debug server, everything seems to be fine. You can even connect with an Eclipse client to the server. However, if you try to follow the description in the Extensions Workshop and you try to connect with a browser to https://localhost:7443/jazz/admin you see a screen like the one below.

SSL Error_1

In Chrome the error claims

This site can’t provide a secure connection localhost sent an invalid response.

Try running Windows Network Diagnostics.

ERR_SSL_PROTOCOL_ERROR

Firefox is even more alarming and says

Secure Connection Failed

An error occurred during a connection to localhost:7443. Peer reports it experienced an internal error. Error code: SSL_ERROR_INTERNAL_ERROR_ALERT

The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.

Please contact the website owners to inform them of this problem.

Learn more…

Report errors like this to help Mozilla identify and block malicious sites

Edge complains

Can’t connect securely to this page

This might be because the site uses outdated or unsafe TLS security settings. If this keeps happening, try contacting the website’s owner.

Try this:

Go back to the last page

It looks like the usual SSL issue with the standard certificate, but it actually is not. It does not allow you to proceed and add an exception to connect to the web site.

You are basically stuck.

I had the same issue yesterday with 6.0.5 and 6.0.6 and I knew it was working very recently – when I moved to my new laptop. I searched the internet and was not really sure about a solution still. The answers pointed at version issues and certification mismatches. So I decided to switch the Eclipse environment to a current JDK and that solved the problem. I have tried it with RTC 6.0.5 and 6.0.6 but I am pretty sure it would work with other versions as well.

Please note that the Extensions workshop suggests to use the same JDK the Server uses. I think since Browsers have recently increased their security measures, this is no longer true.

Solution

Here how to fix the problem.

  • Download and install a recent Java JDK/JRE that is compatible with the version of the RTC SDK you are working with.
    • 6.0.6 is compatible with Java 1.8
    • I use a JDK because there are other development tools that require a JDK
  • Open the Eclipse server development workspace
  • Open Window>preferences
  • Type JRE and navigate to Java>Installed JREs
    • Add the new JDK/JRE
    • Select the JDK/JRE as active click Apply and CloseNewJDK
    • Navigate to Java>Installed JREs>Execution Environments
    • Select the Execution Environment with the matching version
    • Select the new JDK/JRE as active click Apply and CloseExecEnvironment
  • Launch the Jetty debug serverLaunch

Now try to login to https://localhost:7443/jazz/admin again. You still get an error, however the browser provides the Advanced link to continue connection to the site.

SSL Error_2

The different browsers display it different, but the concept is the same for all of them.

Summary

Recent increases of browser security checks and invalidation of certificates requires to update to recent versions of the Java JDK/JRE to be used with the Extensions workshop. As always I hope that this helps people out there.

Status History Presentation for RTC

Rational Team Concert (RTC) has a built-in feature to view the history of a work item. But especially for work items with many changes, it is hard to follow the Status of a work item over time. That’s why Lukas created the amazing Status History Presentation for RTC and published it in the DACH Jazz Community project. It shows all Status changes since the creation of the work item in form of a timeline in the RTC work item Web UI.

Follow the instructions in the Status History Presentation for RTC project to download and install the editor presentation server extension. The Open Source project also serves as a great example for how to write a custom editor presentation.

After installing this extension plug-in into your RTC instance (server side), you will be able to add a “Status History” presentation on your work item editors (WEB only). You can add this  presentation to a work item editor section for example the Quick Info section. Once the presentation is available it will

  • Show all state changes in a timeline
  • Show who did the state change
  • Show the number of days a work item was or is in a state
  • A Rich Hover shows additional information about the changes made together with the state change

Many thanks to the DACH Jazz Community for sharing their amazing work.

Query Models or how to find stuff with the RTC Java API

Although I have done my share using and blogging about the API there are still a lot of uncharted areas. How can I use the RTC API to find a user by the name? This is a question that came up recently in the forum and is one of many questions, I did not have a good answer until today.

While writing my last post, for whatever reason I started thinking about this question. I decided to have a quick peek and try to find out. As a result this blog post describes how questions like that can be approached with the RTC Java API.

The Problem

The API provided for objects such as contributors, build results and a lot more model elements used in the RTC application is not necessarily the API a human would expect. This is because the API is written to make it easy to develop the tool and not to make it easy for a human to access data. So a question “How do I find a user by the name” is not necessarily something the RTC API would be optimized for.

If a user logs into RTC, they provide the ID and not the name. After login the user ID is available in the API and that is the glue used for almost all internal computation. The user name is usually nothing of interest to the API. However there are cases, for example integration scenarios, where questions like this might be of interest. So how does RTC solve this under the covers?

Work Item Queries

Please don’t confuse the Query Models in this post with work item queries and work item expressions. To search for work items see

License

The post contains published code, so our lawyers reminded me to state that the code in this post is derived from examples from Jazz.net as well as the RTC SDK. The usage of code from that example source code is governed by this license. Therefore this code is governed by this license. I found a section relevant to source code at the and of the license. Please also remember, as stated in the disclaimer, that this code comes with the usual lack of promise or guarantee. Enjoy!

Just starting with extending RTC?

If you just get started with extending Rational Team Concert, or create API based automation, start with the post Learning To Fly: Getting Started with the RTC Java API’s and follow the linked resources.

You should be able to use the following code in this environment and get your own automation or extension working.

Compatibility

This code has been used with RTC 6.0 and is prepared to be used with RTC 6.0.x with no changes and it is pretty safe to assume, that the code will work with newer versions of RTC. It should however run with any version of RTC that has the specific API already implemented. The code shown below should work with almost all versions of RTC.

The post shows client, common and server API that are available in the RTC Server SDK and the RTC Plain Java Client Libraries.

Download

You can download the Eclipse project with the examples.  Please note, there might be restrictions to access Dropbox and therefore the code in your company or download location.

Solution

RTC works against a database. Some domains such as work items and SCM provide higher level query mechanisms to support user configurable queries in the UI to find stuff. Other domains do not.  But under the cover, the RTC API provides query mechanisms to query the database for data and to manage result sets.

The RTC API provides a common query service and  query models to be able to define and run queries for a wide variety of RTC objects.  The query service and the query models are common API and can be used in Java client applications, Eclipse client extensions as well as in RTC Server extensions.

Although there are questions and examples in the forums and there are some examples like this wiki page and somewhat hidden in this article (section Querying for Items), there is no good description how this works on a broader level. Needless to say that there is no description how to find the entry points into the API either. This post tries to help as good as possible.

How to find the query models

The best way to approach this is to setup a RTC Development environment by following the getting started post. Thsi means follow Setting up Rational Team Concert for API Development and the Extensions workshop and perform at least lab 1. Now you have an Eclipse with a RTC SDK set up that provides you with searchable example code. Without this environment, it is pretty pointless to try to approach this API.

You can use the naming conventions used while developing RTC to search for the query models. In the Plug-in Development Perspective select the menu Search>Java. Type *QueryModel as search string. Select declarations and the other choices shown below, then click Search.

searchquerymodel

Be patient while the SDK is searched. Dependent on the version of RTC you should finally see a search view similar to the one below.

searchquerymodelresult

Note that more than 960 query modes are found. There might be some duplicates and some might be totally uninteresting but there is obviously an enormous potential to access data in RTC.

If you know the model element interface you are interested in, for example an IBuildResult,you can use these approaches to find the related query model.

You can try to use the Eclipse content assist capability of the Plugin Development Environment (PDE) to find the related query model. Type in the name of the model element without the leading I and append QueryModel to it. To find the query model for IBuildResult, type BuildResultQueryModel. While you type use Ctrl+Space for content assist. The PDE should find the query model class.

contentassistpde

You can also use the search approach from earlier to search for the specific class BuildResultQueryModel with or without using the asterisk.

specificsearch

That way the Eclipse client, if set up correctly, allows to find the query model for the model element interfaces you are interested in.

What is provided by the query models?

The query models allow to define queries, to find model elements related to the query model. The query model provides comparisons, Boolean operations on properties specific to the model elements, sort and filter operations. The queries can be constructed and called with parameters. The queries can then be run using a query service and return a result set that can be further processed.

Example 1: Find the build results related to a specific build definition that are tagged with a specific tag.

Example 2: Find a Contributor by user name.

The entry point of a query model is the ROOT entry of the query model. It allows to instantiate queries against the query model and defines the interfaces available for the specific query model. These interfaces also specify which properties an object has, how to access them and the Boolean and other operations available to operate on this model element.

The image below shows the query model root BuildResultQueryModel.ROOT. It returns an implementation class. The BuildResultQueryModel interface also extends interfaces BaseBuildResultQueryModel and ISingleItemQueryModel.

querymodelroot

The interface BaseBuildResultQueryModel defines which properties the model element IBuildResult exposes in queries.

buildresultquerymodel

ISingleItemQueryModel defines operators such as equals or contained in providing a IPredicate interface.

isingleitemquerymodel

The IPredicate interface provides the interfaces to create the Boolean operations and, or, not.

ipredicate

Given this pattern, it is possible to create complex expressions.

Using the query models

Using the Query mechanism typically works in the following steps.

Create a query for the QueryModel for the model element (example [ModelElementName]=BuildResult.

IItemQuery query = IItemQuery.FACTORY.newInstance([ModelElementName]QueryModel.ROOT);

Create a predicate to filter the results based on some properties. This specific example uses a parameter of type string that gets a value passed. Instead of the paramter, it would also be possible to hard code a string here.

IPredicate predicate = [ModelElementName]QueryModel.ROOT.property()._eq(query.newStringArg());

Use the predicate from the step before as filter for the query.

IItemQuery filtered = (IItemQuery) query.filter(predicate);

Finally use the query service to run the query. Here a parameter of type string is passed to the query. Result sets can be big, so the last parameter is used to pass how many results should be retrieved.

IItemQueryPage page = queryService.queryItems(filtered, new Object[] { "Jerry Jazz"}, 1 );

All this can be as compact as in the following example.

IItemQueryPage page = queryService.queryItems(IItemQuery.FACTORY.newInstance(IterationPlanRecordQueryModel.ROOT), IQueryService.EMPTY_PARAMETERS, IQueryService.DATA_QUERY_MAX_PAGE_SIZE);

Get contributor by user name

Lets look at how the code for this forum question.: “How can I get the contributor for a user if I have the user name and not the ID?”. The code is inspired by very similar code that is used for the RTC Jabber integration.

We are looking for an IContributor. So the looking for the query model seems to be ContributorQueryModel.

First the code creates the query for the ContributorQueryModel. Then it creates a predicate to filter out a contributor with a specified name. The predicate uses an argument for the user name instead of providing the user name already here as string. The predicate is set as filter.

This is plain java client library code. There is no direct access to the com.ibm.team.repository.common.service.IQueryService. To get the IQueryService the code uses a trick. the IQueryService is available from the Implementation Class for ITeamRepository.  The teamRepository object is casted to com.ibm.team.repository.client.internal.TeamRepository. This makes the usage of QueryModels unsupported due to using unsupported internal code.

The IQueryService is indirectly available in some client libraries as well.

ITeamBuildClient buildClient = (ITeamBuildClient) teamRepository.getClientLibrary(ITeamBuildClient.class);
IItemQueryPage queryPage = buildClient.queryItems(query, parameters, IQueryService.ITEM_QUERY_MAX_PAGE_SIZE, monitor);

Once the QueryService is available the query is executed. In the example the user name is passed as a new parameter to the query. The result expects no more than one result.

The paged result is retrieved as list of handles and further processed.

// Create a query for the ContributorQueryModel
final IItemQuery query = IItemQuery.FACTORY.newInstance(ContributorQueryModel.ROOT);
// Create a predicate with a parameter to search for name property  
final IPredicate predicate = ContributorQueryModel.ROOT.name()._eq(query.newStringArg());
// Use the predicate as query filter 
final IItemQuery filtered = (IItemQuery) query.filter(predicate);
// Get the query service. This is a cast to an internal class. Note TeamRepository and not ITeamRepository is casted.
final IQueryService qs = ((TeamRepository) teamRepository).getQueryService();
// Run this ItemQuery. Note, there are also other types of queries qs.queryData(dataQuery, parameters, pageSize)
final IItemQueryPage page = qs.queryItems(filtered, new Object[] { findUserByName }, 1 /* IQueryService.DATA_QUERY_MAX_PAGE_SIZE */);
// Get the item handles if any
final List<?> handles = page.getItemHandles();
System.out.println("Hits: " + handles.size());
if (!handles.isEmpty()) {
	System.out.println("Found user.");
	// Resolve and print the information to the contributor object.
	final IContributorHandle handle = (IContributorHandle) handles.get(0);

Please note that there are various operations available to create the predicate and that the predicate can actually more complex. The code above uses the method

_eq()

to check for an equal string. In this case the match has to be exact.

Another option would be to search with ignore case option. This can be done using the following predicate definition:

final IPredicate predicate = ContributorQueryModel.ROOT.name()._ignoreCaseLike(query.newStringArg());

The complete code can be found below and is available for download. The downloadable code contains the example to search for the user by exact (case sensitive) user name and an example to search for the contributor by e-mai with a case insensitive match.

If the QueryUserByName example class is called with the required parameters like.

"https://clm.example.com:9443/ccm" "myadmin" "myadmin" "John Doe"

the result looks like this, provided the user exists, of course.

executionresult

Differences in the Server API

Server extensions must extend com.ibm.team.repository.service.AbstractService. This allows to use com.ibm.team.repository.service.AbstractService.getService(Class) to get the IQueryService in a server Extension like this:

IQueryService queryService = this.getService(IQueryService.class);

The rest of the API is as described above.

Dynamic Query  Model

There is also a dynamic query model based on the IItemType. It can be created as shown below.

IDynamicItemQueryModel dynamicQueryModel = IBuildResult.ITEM_TYPE.getQueryModel();

From the documentation in the code:

Generally, static query models should be used whenever they are visible and the types/properties are known at compile time. Note also that there are no API contracts regarding dynamic APIs – model objects may change shape, queryable properties, etc.”

So use the static version as described above.

Interesting examples in the SDK

Search for “Owned By” is in com.ibm.team.repository.client.tests.query.ExistsPredicateQueryTests.testExistsPredicateUsingLinks()

The full example code

Please find below the complete code for the RTC Plain Java Client Library version to find a contributor by its user name.

/*******************************************************************************
 * Licensed Materials - Property of IBM
 * (c) Copyright IBM Corporation 2017. All Rights Reserved. 
 *
 * Note to U.S. Government Users Restricted Rights:  Use, duplication or 
 * disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
 *******************************************************************************/
package com.ibm.team.workitem.ide.ui.example;

import java.util.List;

import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.NullProgressMonitor;

import com.ibm.team.repository.client.IItemManager;
import com.ibm.team.repository.client.ITeamRepository;
import com.ibm.team.repository.client.ITeamRepository.ILoginHandler;
import com.ibm.team.repository.client.ITeamRepository.ILoginHandler.ILoginInfo;
import com.ibm.team.repository.client.TeamPlatform;
import com.ibm.team.repository.client.internal.TeamRepository;
import com.ibm.team.repository.common.IContributor;
import com.ibm.team.repository.common.IContributorHandle;
import com.ibm.team.repository.common.TeamRepositoryException;
import com.ibm.team.repository.common.model.query.BaseContributorQueryModel.ContributorQueryModel;
import com.ibm.team.repository.common.query.IItemQuery;
import com.ibm.team.repository.common.query.IItemQueryPage;
import com.ibm.team.repository.common.query.ast.IPredicate;
import com.ibm.team.repository.common.service.IQueryService;

/**
 * Uses the ContributorQueryModel to search for a user by the user name 
 * and not the ID.
 * 
 * 
 * Example code, see
 * https://jazz.net/wiki/bin/view/Main/ProgrammaticWorkItemCreation.
 */
public class QueryUserByName {

	private static class LoginHandler implements ILoginHandler, ILoginInfo {

		private String fUserId;
		private String fPassword;

		private LoginHandler(String userId, String password) {
			fUserId = userId;
			fPassword = password;
		}

		public String getUserId() {
			return fUserId;
		}

		public String getPassword() {
			return fPassword;
		}

		public ILoginInfo challenge(ITeamRepository repository) {
			return this;
		}
	}

	public static void main(String[] args) {

		boolean result;
		TeamPlatform.startup();
		try {
			result = run(args);
		} catch (TeamRepositoryException x) {
			x.printStackTrace();
			result = false;
		} finally {
			TeamPlatform.shutdown();
		}

		if (!result)
			System.exit(1);
	}

	private static boolean run(String[] args) throws TeamRepositoryException {

		if (args.length != 4) {
			System.out
					.println("Usage: QueryContributorByName [repositoryURI] [userId] [password] [NameOfUserToSearch]");
			return false;
		}

		IProgressMonitor monitor = new NullProgressMonitor();
		final String repositoryURI = args[0];
		final String userId = args[1];
		final String password = args[2];
		final String findUserByName = args[3];
		ITeamRepository teamRepository = TeamPlatform
				.getTeamRepositoryService().getTeamRepository(repositoryURI);
		teamRepository.registerLoginHandler(new LoginHandler(userId, password));
		teamRepository.login(monitor);

		/***
		 * There is a wide variety of query models available for several domains that allow to query 
		 * the elements and filter the results.
		 *
		 * For some examples on the topic
		 * @see https://jazz.net/wiki/bin/view/Main/QueryDevGuide#ExampleOne
		 * @see https://jazz.net/library/article/1229
		 */

		// Create a query for the ContributorQueryModel
		final IItemQuery query = IItemQuery.FACTORY.newInstance(ContributorQueryModel.ROOT);
		// Create a predicate with a parameter to search for name property  
		final IPredicate predicate = ContributorQueryModel.ROOT.name()._eq(query.newStringArg());
		// Use the predicate as query filter 
		final IItemQuery filtered = (IItemQuery) query.filter(predicate);
		// Get the query service. This is a cast to an internal class. Note TeamRepository and not ITeamRepository is casted.
		final IQueryService qs = ((TeamRepository) teamRepository).getQueryService();
		// Run this ItemQuery. Note, there are also other types of queries qs.queryData(dataQuery, parameters, pageSize)
		final IItemQueryPage page = qs.queryItems(filtered, new Object[] { findUserByName }, 1 /* IQueryService.DATA_QUERY_MAX_PAGE_SIZE */);
		// Get the item handles if any
		final List<?> handles = page.getItemHandles();
		System.out.println("Hits: " + handles.size());
		if (!handles.isEmpty()) {
			System.out.println("Found user.");
			// Resolve and print the information to the contributor object.
			final IContributorHandle handle = (IContributorHandle) handles.get(0);
			IContributor foundContributor = (IContributor) teamRepository.itemManager().fetchCompleteItem(handle, IItemManager.DEFAULT, monitor);
			System.out.println("UUID: " + foundContributor.getItemId());
			System.out.println("ID: " + foundContributor.getUserId());
			System.out.println("Name: " + foundContributor.getName());
			System.out.println("E-Mail: " + foundContributor.getEmailAddress());
			System.out.println("Archived: " + foundContributor.isArchived());
		}			
		teamRepository.logout();
		return true;
	}
}

Additional Examples

Summary

As always I hope this helps someone out there with running RTC. Please keep in mind that this is as usual a collection of very basic examples with no or very limited testing and error handling. Please see the links in the getting started section for more examples, especially if you are just getting started with the RTC APIs.

Is The Extension Deployed? How Can I Redeploy?

This is a question that comes up every so often in the forums. Unfortunately there is no place, I am aware of, where this really gets explained and it seems to require the knowledge of mysterious URL’s. Is there an easy way?

I wanted to always blog this, but somehow I just never did. Let’s unveil it now.

Just Starting With Extending RTC?

If you just get started with extending Rational Team Concert, or create API based automation, start with the post Learning To Fly: Getting Started with the RTC Java API’s and follow the linked resources.

You should be able to use the following code in this environment and get your own automation or extension working.

Differences between Tomcat and WebSphere Application Server

The deployment process on Tomcat and WebSphere Application Server (WAS) is basically the same. The folders where you deploy the extension are the same within server/conf/ccm . The mechanisms described below work the same for WAS, except the location of the file built-on.txt.

Internal Tools

The best way to check if a server extension is deployed on a server is to use some Internal Tools. These hidden Internal Tools are also the easiest way to enforce redeployment of the applications and added extensions.

The internal tools can be accessed by injecting the string ?internal=true into the server administration URL for a Jazz application. This makes it available on the Server Administration page. It is possible to append the string above behind the URL for the main administration pages and then open the server administration. If the server administration page is already open and it contains an action the string needs to be injected before the # (hash tag) separating the action from the base URL.

The image below shows the injected string in the server administration page before the action.

InternalTool_Injected

The image below shows the URL when just appending the string before opening a specific server administration page.

InternalTool_1An URL like

 https://<server>:<port>/jazz/admin?internal=true#action=jazz.viewPage&id=com.ibm.team.repository.server

shows the internal tools menu that can then be used to look at data that is usually not revealed.

Use the Component Status menu action to check if an extension is deployed. It opens the Component Status page with all components that are deployed. You can then use the browsers find functionality to search for your extension, or rather the name of the component you chose. If you followed my advice and chose a unique and easy naming schema, you should be able to find it. The image below shows the example of the RTC Extensions Workshop at the end of the list. Searching for rtcext revealed this in no time.

DeployedComponent

If your component does not show up, fix your deployment and try again.

Use the Server Reset menu action to trigger a re-provisioning of all extensions. Reboot the server after pressing the button. If a new version of an extension is deployed, this makes sure that the cache gets deleted and all the features, including the new version, are re-read.

Server Patch Extension

** Update ** Eric suggests another way to be able to see your extension in the comments below. Use the Server Patch Extension – https://jazz.net/wiki/bin/view/Main/ServerPatchExtension so that your plugin is listed on the CCM Admin Page

“Undeploy” or Deploy a Fix or New Version

To get rid of custom extensions, it is necessary to remove the files that belong to the deployed extension and then you have to force a server reset and redeploy. The files that typically belong to a server extension are

  1. The provision profile file. This is a file with extension *.ini  located in the  server\conf\ccm\provision_profiles folder, created for the extension that specifies the feature and its site location
  2. The extension specific site folder usually located in the folder server\conf\ccm\sites  that contains the site.xml and the plug-ins and features in sub folders

To update or deploy a new version you can:

  1. Deploy the new version (overwrite the existing files mentioned above)
  2. Force a server reset and restart the server

This works if the file names and the structure does not change. If you are not sure and you don’t want to leave any remnants you can

  1. Undeploy the old version by removing all deployed files and folders
  2. Deploy the new version
  3. Force a server reset and restart the server

This is the most secure approach.

Force Server Reset and Redeploying using Internal Tools

The runtime of the Jazz servers caches information about deployed applications. If a new version of an application is deployed, the server would not pick that up and would not redeploy it. To enforce redeploying, it is necessary to request a server reset. There are several ways to do that.

One way is to use the Internal Tools and click on the Server Reset menu action. On the page displayed, click the Request Server Reset button. Next time the server is started, all plugins are redeployed.

Force Server Reset and Redeploying – Alternative Approach

If your server is not up, another way to enforce a server reset is to search for the file built-on.txt in the work folder of your application server. If this file is deleted, a server reset is performed the next time the server starts. For Tomcat you can find the file in the work folder of the application.

Built-on

For the traditional WebSphere Application Server you should find the file located somewhere underneath the profile folder.

For the WebSphere Liberty Profile application server, you should find the file located somewhere underneath the workarea folder. E.g. in [Install Dir]\JazzTeamServer\server\liberty\servers\clm\workarea\.

Please note that the built-on.txt file is also located in WEB-INF folder in the WAR file and in the WEB-INF folder of the deflated war file. Don’t delete these files.

Summary

The Internal Tools provide an easy way to find out if a custom component is deployed and to request a server reset.

As always I hope this helps practitioners out there to be more effective.