The RTC Extensions Workshop has been updated for EWM 7.0.x

I am very passionate about the RTC Extensions Workshop as you might be able to tell from the content of this blog. Performing it with EWM 7.0.x provided several challenges. It became apparent that an update to the workshop would be beneficial.

I spent a considerable amount of time in the past two months to update the workshop. As a summary the following items where addressed:

  1. Since the CCM server is shipped with WebSphere liberty profile, configuring the server for debugging needed to be changed. The old way to configure the server still worked in the 6.0.x versions, so it went unnoticed. With EWM 7.0.1 this is no longer the case and the workshop was updated to address this.
  2. The advanced capabilities introduced in the EWM SCM system in the 6.x and later caused a deviation of the screen shots showing the pending changes. The workshop setup tool was slightly changed to fix this.
  3. The workshop setup tool and its shell script has been tested with Linux and MAC OS.
  4. I wanted to add a section to Lab 1, explaining how to setup the existing Eclipse client/server development workspaces to better support development and debugging of the Plain Java Client Libraries forever. The new last optional section addresses this. For this reason Lab 1 of the workshop is a must for anyone intending to create Java based automation or extensions to RTC/EWM.
  5. I had an errata list with a number of small issues, typos, naming inconsistencies and the like that were fixed. During reviews a bunch more showed up and were fixed.
  6. A colleague ran the workshop on his MAC, so this works. Use whatever is available for MAC like Eclipse and where this is not specifically available, use the Linux versions.

The RTC Extensions Workshop has been published with an additional section for the new EWM versions and is now available for download. I will update recent posts around the workshop in the next few days.

As always, I hope that this blog post helps the users in the Jazz Community.

EWM/RTC Plain Java Client Libraries and Maven

I have worked with Maven applications using the RTC Plain Java Client libraries often recently. It has not always been fun, I have been struggling with Maven. I have been struggling with the Plain Java Client Libraries and Maven interacting. In this bog, I will try to share some things I have learned over the time, and recently, that you can use when working with Maven, especially in the context of EWM/RTC automation using the Plain Java Client Libraries.

Maven basics

How to install Maven is documented in this link. I do not intent to rewrite this documentation. I just want to emphasize some things that I always run into.

  1. Using Maven, requires a Java JDK. A JRE is not sufficient. Where to download a valid, supported, IBM JDK, is explained below.
  2. When using Maven in the command line, make sure to have the JAVA_HOME set to the correct JDK.
  3. When using Maven in the command line, make sure to have Maven added to the path.
  4. When using Maven with the Eclipse IDE, make sure to have an up to date Maven installed. I usually download the recent version, install it to work with a command line and then add that install to Eclipse.
  5. When using Maven with the Eclipse IDE, there is still the need to have a compatible JDK configured. Make sure to go to Preferences>Java>Installed JRE’s, add the JDK and set it as active. Then go to Preferences>Java>Installed JRE’s>Execution Environments configure an environment with the JDK. E.g. JavaSE-1.8 for EWM 7.0.x.
  6. When using Maven for a project in an Eclipse IDE right click on the project
    1. Use Maven>Update Project to make sure Eclipse is configured correctly.
    2. Use Run As>Maven clean on the Project or the pom.xml.
    3. Use Run As>Maven install on the Project or the pom.xml.
    4. Sometimes the Eclipse build mechanism seems to be in conflict with Maven, Project>Clean might help.
  7. Sometimes weird things happen. Run maven with -e or -X to see more details.
  8. Sometimes weird things happen, if all fails, consider to delete the local repository folder in the .m2 folder in your user directory.

Maven and the EWM Plain Java client libraries

When working with EWM/RTC automation based on the plain java client libraries, it is possible to download a valid, supported, IBM JDK on the same page where the Plain Java Client Libraries are hosted. When working with EWM/RTC and the Plain Java Client Libraries, the items below show the most important files to download to work with the Plain Java Client Libraries. To get them go to the All downloads tab for the version required. As an example for EWM 7.0.2 go to https://jazz.net/downloads/workflow-management/releases/7.0.2?p=allDownloads.

  1. Get the 10-Free Developers License Activation Kit. Scroll down to the section License Keys and download the 10-Free Developers License Activation Kit. It is very useful to run EWM test systems.
  2. Get the Client for Eclipse. Scroll to the Plain .zip Files section. Download the Client for Eclipse for your architecture.
  3. Get the plain Java Client Libraries. In the Plain .zip Files section download the Plain Java Client Libraries and the API documentation.
All Downloads page

The JDK

The download for the EWM Client for Eclipse contains the required JDK in the folder jazz/jdk. Unpack the EWM client and use the JDK for your development with the Eclipse Client Libraries and Maven.

The trouble with Maven and the Plain Java Client Libraries.

Unfortunately there is no Maven library for the Plain Java Client Libraries. This is a problem, because there is no real good solution (that I could find) to work with Maven and libraries that do not have a maven repository.

In some projects, I was able to get away with working with Maven and the Plain Java Client Libraries, using an Eclipse User Library with the Plain Java Client Libraries. This works, if at all, only in the Eclipse IDE.

In a recent project this solution was not manageable. I was unable to compile, or I was able to compile, but running the application failed with class not found exceptions. I got it working on my laptop, but we were unable to get it working for a colleague. We lost days.

My usual strategy of using <put your most popular internet search engine here> to find solutions did not really help. Solutions that I found where cryptic, not understandable and/or did not work.

Packaging the Plain Java Client Libraries as local Maven repository.

I finally found this script https://github.com/cokeSchlumpf/mvn-rtc-java-api that allows packaging the plain java client libraries as a maven repository and include that in the application POM file. This allows shipping the Plain Java Client Libraries as Maven repository with the application and its code to be able to run Maven in Eclipse as well as on the command line.

Although the script finally was a life saver, I had some tough learning to get things right. The main issue was that it is a Bash script and my Windows system does not Bash. I tried installing Cygwin64 but realized that I would have to install all kind of stuff to get that working. As I had recently done some other EWM and containerization related work on my NAS, I ended up setting up the required Maven and JDK on my NAS system. I was able to run the script from there 7Zip the result and upload it to my development environment.

In my first attempt, unfortunately, I did not pay enough attention to the format of the dependencies. The dependencies to external libraries has the format <groupId>:<artifactId>:<version>.

# 6.0.2
export DEPENDENCIES="org.apache.james:apache-mime4j:0.6 commons-io:commons-io:1.2 org.apache.httpcomponents:httpclient:4.5 org.apache.httpcomponents:httpclient-cache:4.5 org.apache.httpcomponents:httpclient-win:4.5 org.apache.httpcomponents:httpcore:4.4.1 org.apache.httpcomponents:httpcore-ab:4.4.1 org.apache.httpcomponents:httpcore-nio:4.4.1 org.apache.httpcomponents:httpmime:4.5"

For whatever reason I did not realize that and only provided the <groupId>. This creates an invalid repository. The symptom is the warning message ‘The POM for …. is invalid, transitive dependencies (if any) will not be available’. It took a while to figure out what the reason was. I ended up recreating the repository using the following dependencies values for 7.0.1:

# 7.0.1
export DEPENDENCIES="org.apache.james:apache-mime4j:0.6 commons-io:commons-io:1.2 org.apache.httpcomponents:httpclient:4.5.6 org.apache.httpcomponents:httpclient-cache:4.5.6 org.apache.httpcomponents:httpclient-win:4.5.6 org.apache.httpcomponents:httpcore:4.4.10 org.apache.httpcomponents:httpcore-ab:4.4.10 org.apache.httpcomponents:httpcore-nio:4.4.10 org.apache.httpcomponents:httpmime:4.5.6"

It is totally possible that some of my attempts to fix the above problem made things more miserable. I just realized the severity of the issue when I merged my projects to one and still had the problem.

Please note, if you follow the script above, provide the repository and change the pom of your application and run

mvn clean install

the repository is copied into the local maven repository on your machine. It is stored in the user directory in the folder

.m2/repository/com/ibm/rtc/rtc-java-api

If you build a new version of the repository, make sure to delete the folder above, to make sure your new version is actually installed. Maven does not recognize a difference and will not replace the local files on its own. The reason is that there is no version change visible in the data. This holds true also for the machines of your fellow developers. So when recreating a new Plain Java Client Libraries maven repository with the script, make sure to delete the old local maven repository for the library rtc-java-api in you local .m2 folder as explained above. Then run

mvn clean install

This will recreate the maven library.

I finally succeeded with running

# 7.0.1
export DEPENDENCIES="org.apache.james:apache-mime4j:0.6 commons-io:commons-io:1.2 org.apache.httpcomponents:httpclient:4.5.6 org.apache.httpcomponents:httpclient-cache:4.5.6 org.apache.httpcomponents:httpclient-win:4.5.6 org.apache.httpcomponents:httpcore:4.4.10 org.apache.httpcomponents:httpcore-ab:4.4.10 org.apache.httpcomponents:httpcore-nio:4.4.10 org.apache.httpcomponents:httpmime:4.5.6"

./setup-repo.sh -l ./libs-7.0.1 -r ./repo -d "${DEPENDENCIES}" -v 7.0.1

I followed the readme and added the following to my pom.xml

	<!-- The maven dependency for the EWM Plain Java Client Libraries provided in a local repository -->
		<dependency>
			<groupId>com.ibm.rtc</groupId>
			<artifactId>rtc-java-api</artifactId>
			<version>7.0.1</version>
			<type>pom</type>
		</dependency>

and the repository:

<!-- The maven repository for the EWM Plain Java Client Libraries -->
<repositories>
    <repository>
        <id>project-repo</id>
        <releases>
            <enabled>true</enabled>
            <checksumPolicy>ignore</checksumPolicy>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
        <url>file://${project.basedir}/repo</url>
    </repository>
</repositories>

The location is based on a project property:

		<project.basedir>${basedir}</project.basedir>

So the files of the repository where copied and located in my Eclipse project in the folder repo in the project root folder.

The repository is bundled in the folder repo

Now it is possible to run Maven from within Eclipse as well as on the shell/command line and my project cleans, installs and liberty:run and the warning mentioned above is gone.

I can only suggest to use the mvn-rtc-java-api – Simple setup RTC Java Client API for Maven project. I would like to thank Victor and Michael for their great contribution. I can only suggest to use it for your purposes.

Whats Next?

I am a bit hesitant to use my personal NAS system as an extended Linux system for development. I will see if I can find some time to understand what the application does in detail and maybe rewrite it in Python. That would make it more operating system independent and allow broader usage.

Summary

As always, I hope that this post helps users out there to achieve their goal and saves some hours of work.

Issues with the EWM/RTC Extensions workshop in 7.0.x versions

I have seen issues with the RTC Client SDK for the 7.0.x releases recently. The Server SDK seems not to be affected.

Client plug-in development

UPDATE: The issue with the Client SDK has been found and corrected. Just download and use the new Client SDK.

UPDATE: The issue with the server debugging can be solved as explained below.

If you are using the  Rational Team Concert Extensions Workshop for the EWM/RTC versions 7.0.0, 7.0.1, 7.0.2 and try to start an Eclipse client from the client launch as described in section 1.9 Test the RTC Eclipse client launch, you might run into a problem. I reported the issue in Defect 511733 (use your Jazz.net credentials).

This issue has been solved and you should not see it any longer. Make sure to download the latest Client SDK from Jazz.net.

If you need to develop client extensions and run into this problem, check the status of the defect. Worst case, use a 6.0.6.1 SDK to develop your extension. This is absolutely possible if you do not rely on a new client API feature from 7.0.x.

Server debugging

If you are using the  Rational Team Concert Extensions Workshop for the EWM/RTC versions 7.0.2, trying to launch the development server in debug mode that allows to attach an Eclipse debugger to that server, the server startup fails. In ‘1.2__15 start the server using server.startup.bat’ after adding the debug configuration, the server does not start. The log folder only contains the files start.log and console.log. The content indicates that the connection to the debug port failed because the port was already in use. When checking the pot usage, the port is not in use. This has not been seen in any version before. See Defect 527867 for work arounds and solutions.

Update: see http://EWM Extensions Workshop remote debugging in 7.0.x for how to deal with this.

Adding custom commands to the SCMUtils

The SCMUtils is based on a framework that I developed over time. The framework provides easy to reuse mechanisms to create custom commands that handle most of the essential requirements automatically. It handles calling a command based on a name string, the parameters the command needs and optional parameters. The command can define its own help content that is printed when either the command was called with missing parameters or if the command name is not supported. This blog post explains the steps needed.

What are the SCMUtils?

Please see SCM Utils – SCM data secure sharing, statistics and more for what they are and how to get them.

Enhancing the tool with own commands

It is easy to implement your own commands. The framework project provides abstract classes that can be used to implement own commands.

The framework allows easy implementation of new custom commands.

The class SampleCommandCmd is a simple example that can be used as a starting point. It implements all the basic capabilities that are needed.

The class AbstractCommand can be extended to a simple command that can execute your own code.

The class AbstractTeamrepositoryCommand is an abstract class that can be used to implement custom commands that interact with a Jazz application. It manages the parameter and behavior required for connecting to a Jazz server. In addition it automatically implements scenario logging. It can be extended to implement commands that work against a Jazz server.

It is not necessary to use one of the classes above. It is only required to implement the interface ICommand.

Tips and tricks for implementing custom commands

The framework comes with a sample command, that you can look to understand how the implementation works. If you want to create your own command, you should not do that in the project com.ibm.js.team.supporttools.framework. You should add it in a project area that refers to the com.ibm.js.team.supporttools.framework project. E.g. you can add the command in the project com.ibm.js.team.supporttools.scmutils.

Open the command /com.ibm.js.team.supporttools.framework.SampleCommandCmd and study the class.

Sample class part 1

The class SampleCommandCmd extends AbstractCommand, so it does not inherit logging into a jazz server. It also implements ICommand (which it inherits from AbstractCommand).

The next part is to define a logger. The framework uses Simple Logging Facade for Java (SLF4J) to provide logging. The log file will be written to disc. Use the logger to add custom logging.

The command must implement the default constructor and pass the command id to the framework. The super class implements this, so the command name is passed to the super class.

In addCommandOptions(), the class adds additional options (parameters). The first parameter in addOption() is the option/parameter name, the second defines if the option requires a value. The third parameter is the description of the option.

Sample class part 2

The command has to make sure it gets the options it needs. This is checked in the next step. The method checkParameters() is used to do just this. Test if the required options are available.

The method printSyntax() is called if there is some kind of issue with calling the SCMUtils. Implement your documentation here. The logger by default prints log level info.

Sample class part 3

The entry point for the custom code is the execute() method. This usually gets all the mandatory options and their values. The method getCmd().getOptionValue() is used to get the parameter or option value if the option has a value. The method getCmd().hasOption() is usually if the option does not have a value and is more like a flag.

Then it usually executes the payload code that does whatever the command is supposed to do. The return value should be true if the command succeeded and false otherwise.

Adding the custom command to the CommandFactory

After implementing a new command, it has to be added to the ScmSupportToolsFactory in the project com.ibm.js.team.supporttools.scmutils to make it available to be called..

The command factory

This is where the command needs to be added. See the available commands.

Add a command

Remove the comments before the put(new SampleCommandCmd()), to enable the sample command. You would add a put for your own command here. The SampleCommand is a good learning execise if you want to learn debugging.

Summary

It is really simple to add new commands, there is a set of commands already available, copy one of those if it looks similar to what you want to do.

Like always, I hope this helps someone out there.