I am very passionate about the RTC Extensions Workshop as you might be able to tell from the content of this blog. Performing it with EWM 7.0.x provided several challenges. It became apparent that an update to the workshop would be beneficial.
I spent a considerable amount of time in the past two months to update the workshop. As a summary the following items where addressed:
Since the CCM server is shipped with WebSphere liberty profile, configuring the server for debugging needed to be changed. The old way to configure the server still worked in the 6.0.x versions, so it went unnoticed. With EWM 7.0.1 this is no longer the case and the workshop was updated to address this.
The advanced capabilities introduced in the EWM SCM system in the 6.x and later caused a deviation of the screen shots showing the pending changes. The workshop setup tool was slightly changed to fix this.
The workshop setup tool and its shell script has been tested with Linux and MAC OS.
I wanted to add a section to Lab 1, explaining how to setup the existing Eclipse client/server development workspaces to better support development and debugging of the Plain Java Client Libraries forever. The new last optional section addresses this. For this reason Lab 1 of the workshop is a must for anyone intending to create Java based automation or extensions to RTC/EWM.
I had an errata list with a number of small issues, typos, naming inconsistencies and the like that were fixed. During reviews a bunch more showed up and were fixed.
A colleague ran the workshop on his MAC, so this works. Use whatever is available for MAC like Eclipse and where this is not specifically available, use the Linux versions.
The RTC Extensions Workshop has been published with an additional section for the new EWM versions and is now available for download. I will update recent posts around the workshop in the next few days.
As always, I hope that this blog post helps the users in the Jazz Community.
As mitigation I suggested to replace the JRE with an earlier one, or one that works. Unfortunately, this does not look sustainable. I have tried downloading the newest IBM Java SDK and the issue still happens.
I have looked a little bit deeper into it and discovered that the server.startup script provides a debug option. Using this option allows remote debugging the EWM server. I am looking into an update to the RTC/EWM Extensions Workshop for 7.0.x. until this is available here a procedure how to follow the workshop for 7.0.x versions.
In Lab 1.2, do not add the following lines to the server.startup script, remove these lines:
set JAVA_OPTS=%JAVA_OPTS% -Xdebug
set JAVA_OPTS=%JAVA_OPTS% -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3388
Scroll to the bottom of the server.startup script and inspect the statements behind setting the JAVA_OPTS. Note that there are an action and an option supported.
Special parameters supported for starting Liberty Profile
Start server for remote debugging
The action -debug can be used to start the Liberty Profile Server in debug mode. When following the workshop, continue to Lab 1.5.
Follow the lab until 1.5__32 Add the following steps:
Open a console/cmd window and change directory into the server folder containing the server.startup script. Stop the development server by running server.shutdown.
>server.shutdown
Wait until the server is successfully stopped. Start the development server by running the following command.
>server.startup -debug
Note that the server does not start, it suspends listening for a remote connection on a dt_socket/port. The default socket/port address is 8000.
Server waits in debug console
The server will not continue starting unless a remote connection is opened on that port.
Connect Eclipse as remote debugger
Follow the workshop and open the [RTCExt] Debug Running Liberty Profile launch configuration. This launch is used to connect to a remote Java application. In our case the application is the RTS/EWM development server waiting on socket/port 8000.
Change the port in the launch to 8000 then click Apply and Debug.
Change debug port to 8000 apply and start debugging.
Switch to the command prompt. It takes a while and the server continues to start. Wait for the server application to successfully start.
Continue with the workshop and remote debug the server.
When disconnecting from the remote Java application, the RTC development server will show again, that it waits for a remote connection.
Reprovision the Jazz applications
The option -clean can be used to force to re-provision the Jazz Applications installed on the Liberty Profile server. This is necessary to force rereading custom server extensions during deployment of these extensions. The option can also used together with the -debug action.
Summary
Following the solution described above, allows to successfully perform the Lab 1.5 of the Rational Team Concert Extensions Workshop. I am currently trying to find out if there
Is an option to change the default port for this option
Is an option to avoid suspending the server startup until the remote debugger connection is initially done
I will update the workshop and publish the changes. I have already done several other changes to adjust the workshop for the newer EWM/RTC versions.
As always, I hope that the blog posts here help users in the Jazz community.
While running the test, I found that I had to increase the maximum heap size for the launch [RTCExt] Jetty RTC Server, because the JUnit Test that is run to create the test database, ran out of heap memory. I doubled the value from -Xmx256M to -Xmx512M and the error was gone.
Increase the available heap
The fix is only in the EWM/RTC Client SDK, so it is only necessary to download that again.
Please note that due to the name changes and newer versions of Eclipse, there might be some small differences between the screen shots in the RTC Extensions Workshop and the newer versions. However, they are so minor, it should not be an issue. I am currently looking into updating the current Extensions Workshop material with small changes to help guide the user with these changes.
I have worked with Maven applications using the RTC Plain Java Client libraries often recently. It has not always been fun, I have been struggling with Maven. I have been struggling with the Plain Java Client Libraries and Maven interacting. In this bog, I will try to share some things I have learned over the time, and recently, that you can use when working with Maven, especially in the context of EWM/RTC automation using the Plain Java Client Libraries.
Using Maven, requires a Java JDK. A JRE is not sufficient. Where to download a valid, supported, IBM JDK, is explained below.
When using Maven in the command line, make sure to have the JAVA_HOME set to the correct JDK.
When using Maven in the command line, make sure to have Maven added to the path.
When using Maven with the Eclipse IDE, make sure to have an up to date Maven installed. I usually download the recent version, install it to work with a command line and then add that install to Eclipse.
When using Maven with the Eclipse IDE, there is still the need to have a compatible JDK configured. Make sure to go to Preferences>Java>Installed JRE’s, add the JDK and set it as active. Then go to Preferences>Java>Installed JRE’s>Execution Environments configure an environment with the JDK. E.g. JavaSE-1.8 for EWM 7.0.x.
When using Maven for a project in an Eclipse IDE right click on the project
Use Maven>Update Project to make sure Eclipse is configured correctly.
Use Run As>Maven clean on the Project or the pom.xml.
Use Run As>Maven install on the Project or the pom.xml.
Sometimes the Eclipse build mechanism seems to be in conflict with Maven, Project>Clean might help.
Sometimes weird things happen. Run maven with -e or -X to see more details.
Sometimes weird things happen, if all fails, consider to delete the local repository folder in the .m2 folder in your user directory.
Maven and the EWM Plain Java client libraries
When working with EWM/RTC automation based on the plain java client libraries, it is possible to download a valid, supported, IBM JDK on the same page where the Plain Java Client Libraries are hosted. When working with EWM/RTC and the Plain Java Client Libraries, the items below show the most important files to download to work with the Plain Java Client Libraries. To get them go to the All downloads tab for the version required. As an example for EWM 7.0.2 go to https://jazz.net/downloads/workflow-management/releases/7.0.2?p=allDownloads.
Get the 10-Free Developers License Activation Kit. Scroll down to the section License Keys and download the 10-Free Developers License Activation Kit. It is very useful to run EWM test systems.
Get the Client for Eclipse. Scroll to the Plain .zip Files section. Download the Client for Eclipse for your architecture.
Get the plain Java Client Libraries. In the Plain .zip Files section download the Plain Java Client Libraries and the API documentation.
All Downloads page
The JDK
The download for the EWM Client for Eclipse contains the required JDK in the folder jazz/jdk. Unpack the EWM client and use the JDK for your development with the Eclipse Client Libraries and Maven.
The trouble with Maven and the Plain Java Client Libraries.
Unfortunately there is no Maven library for the Plain Java Client Libraries. This is a problem, because there is no real good solution (that I could find) to work with Maven and libraries that do not have a maven repository.
In some projects, I was able to get away with working with Maven and the Plain Java Client Libraries, using an Eclipse User Library with the Plain Java Client Libraries. This works, if at all, only in the Eclipse IDE.
In a recent project this solution was not manageable. I was unable to compile, or I was able to compile, but running the application failed with class not found exceptions. I got it working on my laptop, but we were unable to get it working for a colleague. We lost days.
My usual strategy of using <put your most popular internet search engine here> to find solutions did not really help. Solutions that I found where cryptic, not understandable and/or did not work.
Packaging the Plain Java Client Libraries as local Maven repository.
I finally found this script https://github.com/cokeSchlumpf/mvn-rtc-java-api that allows packaging the plain java client libraries as a maven repository and include that in the application POM file. This allows shipping the Plain Java Client Libraries as Maven repository with the application and its code to be able to run Maven in Eclipse as well as on the command line.
Although the script finally was a life saver, I had some tough learning to get things right. The main issue was that it is a Bash script and my Windows system does not Bash. I tried installing Cygwin64 but realized that I would have to install all kind of stuff to get that working. As I had recently done some other EWM and containerization related work on my NAS, I ended up setting up the required Maven and JDK on my NAS system. I was able to run the script from there 7Zip the result and upload it to my development environment.
In my first attempt, unfortunately, I did not pay enough attention to the format of the dependencies. The dependencies to external libraries has the format <groupId>:<artifactId>:<version>.
For whatever reason I did not realize that and only provided the <groupId>. This creates an invalid repository. The symptom is the warning message ‘The POM for …. is invalid, transitive dependencies (if any) will not be available’. It took a while to figure out what the reason was. I ended up recreating the repository using the following dependencies values for 7.0.1:
It is totally possible that some of my attempts to fix the above problem made things more miserable. I just realized the severity of the issue when I merged my projects to one and still had the problem.
Please note, if you follow the script above, provide the repository and change the pom of your application and run
mvn clean install
the repository is copied into the local maven repository on your machine. It is stored in the user directory in the folder
.m2/repository/com/ibm/rtc/rtc-java-api
If you build a new version of the repository, make sure to delete the folder above, to make sure your new version is actually installed. Maven does not recognize a difference and will not replace the local files on its own. The reason is that there is no version change visible in the data. This holds true also for the machines of your fellow developers. So when recreating a new Plain Java Client Libraries maven repository with the script, make sure to delete the old local maven repository for the library rtc-java-api in you local .m2 folder as explained above. Then run
I followed the readme and added the following to my pom.xml
<!-- The maven dependency for the EWM Plain Java Client Libraries provided in a local repository -->
<dependency>
<groupId>com.ibm.rtc</groupId>
<artifactId>rtc-java-api</artifactId>
<version>7.0.1</version>
<type>pom</type>
</dependency>
and the repository:
<!-- The maven repository for the EWM Plain Java Client Libraries -->
<repositories>
<repository>
<id>project-repo</id>
<releases>
<enabled>true</enabled>
<checksumPolicy>ignore</checksumPolicy>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<url>file://${project.basedir}/repo</url>
</repository>
</repositories>
The location is based on a project property:
<project.basedir>${basedir}</project.basedir>
So the files of the repository where copied and located in my Eclipse project in the folder repo in the project root folder.
The repository is bundled in the folder repo
Now it is possible to run Maven from within Eclipse as well as on the shell/command line and my project cleans, installs and liberty:run and the warning mentioned above is gone.
I am a bit hesitant to use my personal NAS system as an extended Linux system for development. I will see if I can find some time to understand what the application does in detail and maybe rewrite it in Python. That would make it more operating system independent and allow broader usage.
Summary
As always, I hope that this post helps users out there to achieve their goal and saves some hours of work.
I have seen issues with the RTC Client SDK for the 7.0.x releases recently. The Server SDK seems not to be affected.
Client plug-in development
UPDATE: The issue with the Client SDK has been found and corrected. Just download and use the new Client SDK.
UPDATE: The issue with the server debugging can be solved as explained below.
If you are using the Rational Team Concert Extensions Workshop for the EWM/RTC versions 7.0.0, 7.0.1, 7.0.2 and try to start an Eclipse client from the client launch as described in section 1.9 Test the RTC Eclipse client launch, you might run into a problem. I reported the issue in Defect 511733 (use your Jazz.net credentials).
This issue has been solved and you should not see it any longer. Make sure to download the latest Client SDK from Jazz.net.
If you need to develop client extensions and run into this problem, check the status of the defect. Worst case, use a 6.0.6.1 SDK to develop your extension. This is absolutely possible if you do not rely on a new client API feature from 7.0.x.
Server debugging
If you are using the Rational Team Concert Extensions Workshop for the EWM/RTC versions 7.0.2, trying to launch the development server in debug mode that allows to attach an Eclipse debugger to that server, the server startup fails. In ‘1.2__15 start the server using server.startup.bat’ after adding the debug configuration, the server does not start. The log folder only contains the files start.log and console.log. The content indicates that the connection to the debug port failed because the port was already in use. When checking the pot usage, the port is not in use. This has not been seen in any version before. See Defect 527867 for work arounds and solutions.
The SCMUtils is based on a framework that I developed over time. The framework provides easy to reuse mechanisms to create custom commands that handle most of the essential requirements automatically. It handles calling a command based on a name string, the parameters the command needs and optional parameters. The command can define its own help content that is printed when either the command was called with missing parameters or if the command name is not supported. This blog post explains the steps needed.
It is easy to implement your own commands. The framework project provides abstract classes that can be used to implement own commands.
The framework allows easy implementation of new custom commands.
The class SampleCommandCmd is a simple example that can be used as a starting point. It implements all the basic capabilities that are needed.
The class AbstractCommand can be extended to a simple command that can execute your own code.
The class AbstractTeamrepositoryCommand is an abstract class that can be used to implement custom commands that interact with a Jazz application. It manages the parameter and behavior required for connecting to a Jazz server. In addition it automatically implements scenario logging. It can be extended to implement commands that work against a Jazz server.
It is not necessary to use one of the classes above. It is only required to implement the interface ICommand.
Tips and tricks for implementing custom commands
The framework comes with a sample command, that you can look to understand how the implementation works. If you want to create your own command, you should not do that in the project com.ibm.js.team.supporttools.framework. You should add it in a project area that refers to the com.ibm.js.team.supporttools.framework project. E.g. you can add the command in the project com.ibm.js.team.supporttools.scmutils.
Open the command /com.ibm.js.team.supporttools.framework.SampleCommandCmd and study the class.
Sample class part 1
The class SampleCommandCmd extends AbstractCommand, so it does not inherit logging into a jazz server. It also implements ICommand (which it inherits from AbstractCommand).
The next part is to define a logger. The framework uses Simple Logging Facade for Java (SLF4J) to provide logging. The log file will be written to disc. Use the logger to add custom logging.
The command must implement the default constructor and pass the command id to the framework. The super class implements this, so the command name is passed to the super class.
In addCommandOptions(), the class adds additional options (parameters). The first parameter in addOption() is the option/parameter name, the second defines if the option requires a value. The third parameter is the description of the option.
Sample class part 2
The command has to make sure it gets the options it needs. This is checked in the next step. The method checkParameters() is used to do just this. Test if the required options are available.
The method printSyntax() is called if there is some kind of issue with calling the SCMUtils. Implement your documentation here. The logger by default prints log level info.
Sample class part 3
The entry point for the custom code is the execute() method. This usually gets all the mandatory options and their values. The method getCmd().getOptionValue() is used to get the parameter or option value if the option has a value. The method getCmd().hasOption() is usually if the option does not have a value and is more like a flag.
Then it usually executes the payload code that does whatever the command is supposed to do. The return value should be true if the command succeeded and false otherwise.
Adding the custom command to the CommandFactory
After implementing a new command, it has to be added to the ScmSupportToolsFactory in the project com.ibm.js.team.supporttools.scmutils to make it available to be called..
The command factory
This is where the command needs to be added. See the available commands.
Add a command
Remove the comments before the put(new SampleCommandCmd()), to enable the sample command. You would add a put for your own command here. The SampleCommand is a good learning execise if you want to learn debugging.
Summary
It is really simple to add new commands, there is a set of commands already available, copy one of those if it looks similar to what you want to do.
Last year, I was involved in a customer situation where we had issues with the EWM (RTC) Eclipse client. The customer reported client crashes and also performance/load issues. All these issues where intermittent and very hard to approach. It took a while and a lot of work to get them ironed out.
The performance or client load issue was apparently related to the customers data and its characteristics. I was able to reproduce it at the customer. We needed to be able to reproduce this issue in the development labs. The customer could not share their source code. The only way to share the data was to randomize the information that needed to be protected. Some kind of automation was also required, because we had to handle thousands of SCM components.
So I took some existing source code and wrote a small utility to help providing the data to enable our support reproducing the issue in the lab.
Initial Scope
Initially the tool provided commands supporting the following use cases:
Export the components and their current content from a repository workspace to disc, randomizing the file content but keeping the names and structure intact.
Importing the data into a repository workspace in a different repository recreating the components, their hierarchy and their content from the exported data.
Converting an existing load rule for the original repository to match the component ID’s in the new repository to allow using the load rule.
This allowed us to share the data of the customer in a way that we were able to reproduce the issue in the lab.
To better understand the data, I began to use the tools capability to iterate the information in a repository workspace or stream to collect statistical information. This resulted in commands supporting the following use cases:
Analyze a repository workspace or stream and provide statistics about its components and the contained folders and files.
Analyze all streams in a repository to provide the statistics for each and the totals.
Analyze a sandbox folder and its substructure for the same statistical information.
The information is printed on screen and provided as CSV file, allowing for drill down.
Download and License
The SCM Utils are provided as is, with no warranty or support, under MIT license. The source code is provided here. The repository also contains a comprehensive documentation of the commands and the parameters.
Working with the source code and building
Get the code either downloading the zip file or as local cloned GIT repository. Import all the projects into Eclipse e.g. in the GIT perspective.
Import all projects
There should be three projects as shown below in the Java perspective.
The three projects
The project com.ibm.js.team.supporttools.framework contains a small framework that is used by the tool. The project com.ibm.js.team.supporttools.scmutils contains the code for the SCM Utils. The project ewm-scm-utils contains the readme and all the other projects. I would suggest to close this project.
The projects show errors, because the projects are missing the Plain Java Client Libraries. In addition this tool uses Maven and Maven requires a JDK and an execution environment for it. Make sure to have one installed and activated. The tool was developed with Java 8.
Configure a JDK and set it in the preferences.
JDK is installed and selected
Make sure to activate an execution environment.
Execution Environment configured
Download the plain java client libraries from the All Downloads page of the EWM download. The Doc download for the Plain Java Client Libraries is not needed. Unzip the Plain Java client Libraries into a folder. Now create a user library for the Plain Java Client Libraries. In the preferences navigate to Java>Build Path>User Libraries. Click the New… button. Enter the name PlainJavaApi and click OK. Click on Add External JARs… Browse to the folder where you unpacked the Plain Java Client Libraries. Select all JAR files and click Open. Then click OK.
The PlainJavaApi user library.
There might be still errors, because Maven is missing dependencies.
Open the project com.ibm.js.team.supporttools.framework. Right click on the file pom.xml and select Run As > Maven clean. After that finished right click on the file pom.xml again and select Run As > Maven install. Do the same for com.ibm.js.team.supporttools.scmutils.
Run Maven Clean and Maven Install on both projects.
The code should now compile.
Working tool
The file README.md contains a help/description of the commands and parameters. The tool prints a help when it is run without or wrong parameters.
The code ships with example launches you can explore. To do that click on the triangle close to the bug symbol (or the triangle close to the run symbol). Open the Debug Configurations…. Search for the Java node and unfold it. Select a configuration and check the program arguments.
The available Debug/Run Configuration launches.
You can change the configurations to fit your data or create new ones. When you create new ones, consider storing the launches in a new project. Please do not push your changes to the GIT repository.
Build a runnable jar
The project com.ibm.js.team.supporttools.scmutils contains the file ReadMe – HowToRelease.txt. Follow the description in this file to create a runnable JAR. use the provided batch scripts to make starting the JAR easier.
Note, do not use the option to zip the libraries into the jar file. This results in terrible start times for the tool.
Exporting repository workspaces
The original use case that initiated writing this tool, requires a short explanation. The steps are in general:
Run a exportScmWorkspace to export the data and structure.
Run a importScmWorkspace to import the data where desired.
If using loadrules based on component UUIDs, use convert convertLoadrule to replace the old UUID’s with the new UUID’s.
The command exportScmWorkspace exports the data of a repository workspace. It exports each component in its own zip file. The folder and file structure of the component is kept. The file and folder names are preserved. The file content is randomized by default. The file size is kept, but every character is replaced with a random number. The data is stored in a folder which can be set. In addition to the zip files a JSON file is created that contains the component names, ID’s and information about component hierarchy (sub component).
The optional parameter -exportmode can be used to change the behavior
-exportmode preserve keeps the file content as is.
-exportmode obfuscate replaces the content with random content from a text document containing code lines.
The JSON file contains information required for the import operation.
The structural and content information.
The information can now be used to import the data in any repository, including the repository.
Importing repository workspaces
The information that was exported can be imported into any Jazz SCM system. This is done using the command importScmWorkspace. The command requires the data of a previous export in a folder. In addition it requires a name for a repository workspace to import to, a project area name. The repository workspace is created during the import. The command has two special flags/switches
reuseExistingWorkspace If the workspace to import to already exists, reuse it for import
skipUploadingExistingComponents if a component with the same name already exists, skip the component upload.
These flags have been created to allow recovery from import errors. We ran into out of memory situations that we were unable to resolve. If an import fails with a component, rename the component that failed and remove it from the repository workspace. Then restart the import with these two switches and the import will skip over the components already imported and continue importing the missing components. This can be repeated until the import finally succeeded. Once the import of the components has succeeded, the component hierarchy is recreated.
The parameter componentNameModifier can be used to provide a prefix to be added to the name of each component that is created. This has several benefits. It makes the process repeatable with different prefixes. It also makes it easy to identify which components have been imported.
Example:
-command importScmWorkspace -url "https://elm.example.com:9443/ccm/" -user ralph -password ralph -inputFolder "C:\aTemp\ScmExportRandomize" -workspaceConnection "Imported Hierarchical Workspace" -projectarea "Formal" -componentNameModifier "IBMTestDefault_" -reuseExistingWorkspace -skipUploadingExistingComponents
This shows the following on the console
Importing components create and uploadImport components, recreate component structure
The resulting repository workspace contains all the components and the hierarchy:
Imported components in created repository workspace.
Note that the import stores mapping information between the component UUID’s of the component that are imported and the component that the data was imported in. The data is stored in a JSON file in the input folder.
The UUID mapping for the imported component.The UUID mapping file
Load rule operations
The command convertLoadrule uses the folder containing the information for an export and an import and a load rule file as input and uses the mapping information to replace the source UUID’s in the load rule with the target UUID’s and saves the new load rule.
The command flattenLoadrule converts a load rule that has deep hierarchy removing the hierarchy. It Iterates a load rule and modifies pathPrefix entries for sandboxRelativePath. The modification replaces all / by _ except for the first /. This creates a flat load rule from a load rule that has hierarchy.
The command was created to try to understand if the hierarchy has impact on the load performance.
Analysis of streams and sandboxes
Another set of commands performs an analysis on the structure of repository workspaces, streams and sandboxes. They iterate the object, its content and substructure, and collect statistical information. The information is displayed and stored in CSV files. The CSV files allow to do more analysis later. The analysis currently collects obvious information such as file sizes and number of files per folder and less obvious information such as hierarchy depth, encoding and file extensions.
The following commands are available at the moment:
analyzeScmRepository Iterates the whole repository and analyses the streams it has access to. Can be limited to scopes such as project or team areas. Collects the totals for the scope and uses the the capabilities of the commands below.
analyzeScmWorkspace Analyses one repository workspace or stream.
analyzeSandbox Analyses a folder on the local disk e.g. a sandbox. Excludes special folders such as .jazz3 or .metadata. The list of excluded folders is currently hardcoded.
Example:
-command analyzeScmRepository -url "https://elm.example.com:9443/ccm/" -user ralph -password ralph -outputFolder="C:/aTemp/ScmAnalyzeRepository" -connectionOwnerScope "JKE Banking (Change Management)"
The analysis prints some of the data e.g. for each Stream and each component in the stream.
Collected data for each stream and each component in the stream
The aggregated data is printed for each stream.
Aggregated data for a stream
During the process, the analysis command generates csv files that are stored in the output folder.
CSV files as result
For each stream that is analyzed, a csv file is created. The csv file for the stream has all the data for the components in the stream and aggregated at stream level. The CSV file can be opened with Excel.
The final csv file created named _repository.csv contains the statistics for each stream that was analyzed and aggregated statistics across all analyzed streams. The individual csv files are accessible using a link.
Repository statistics
All csv files that are generated contain multiple sheets.the sheets to the right contain range statistics for the files and the extensions of this files for the analyzed context. The range metrix partitions the ranges in stripes with different top limits (and bottom limits). For each range it counts the the number of files that have a size that fits in a range. It also collects the file extensions associated to files in each range.
Stream operations
An additional set of operations has been recently added.
uploadToStream Uploads a folder and its content as component to a stream and baselines the content. The folder name is used as the component name. The component is created if it does not yet exists. Ownership and visibility of the component is the project area. The component is added to the stream if it is not yet in it. All changes are contained in one change set. When a build result UUID is provided as optional parameter the command will publish the URIs for the stream, the baseline and the component as external links to the build result.
downloadComponentBaseline Downloads the content of a component selected by a baseline into a local file system folder. The component name is created as folder and the content of the component is loaded into that folder.
The code is available, so it can be enhanced. The existing code covers several interesting areas in the RTC SCM API and can be used for inspiration. As always, I hope the SCM Utils will help someone out there.
I ran into an issue related to the JKE Banking Sample application that ships with ELM beginning of this year. I ran into the same issue again. Here is what happens and how to fix it for 6.0.6.1 and for 7.0.
I use the JKE Banking Sample for presenting ELM and for a variety of other reasons that require to look deeper into ELM and how the Jazz tools work and interact. When I start looking into newer versions, I usually install a local test environment first and deploy the JKE Banking Sample. I also define build engines that allow me to test and demonstrate EWM/RTC build. I installed the 6.0.6.1 environment. When I tried to set up build, I ran into an issue. The build was failing with an error “The type java.util.Map$Entry cannot be resolved.” for unknown reasons. I found the issue and published how to solve it in this forum post. I created defect 500981 to address the issue.
The problem was the Eclipse JDT Core Batch Compiler (ECJ) that ships with the sample code was outdated and needed to be replaced with a newer version.
Recently I set up ELM 7.0.0 on my machine and I ran into the same issue.
In the JKE Banking example load a repository workspace to disk. Load the component Build. In the project JKEBuildScripts open the folder libs.
Put the ecj-4.6.3.jar into the folder. Remove the file ecj-3.5.2.jar
Checkin the changes.
In the Build definition go to the Ant tab. If the ANT argument refers to the old ecj-3.5.2.jar, rename the ant arguments to use the new version: -lib ${outputRootDirectory}\${buildLabel}\JKEBuildScripts\libs\ecj-4.6.3.jar
Test the change fixes the problem by running a personal build against that repository workspace.
New ECJ version uploaded.
See the fixed ANT Build Tab of the build definition to refer to the new compiler name in the ANT arguments section.
Ant arguments referring to the correct ECJ compiler.
But replacing the ECJ with a new version did not fix the issue entirely. After a while of trying to solve the issue, it became apparent that somehow the ANT build script was not matching the name for some of the libraries that ship with the sample code. The ANT file referenced file names that had been replaced by libraries that had different versions and different file names. Somehow this was overlooked.
So, in addition to changing the version of the ECJ as described in the link above, It is also necessary to change the line that creates the classpath for building in the build.xml highlighted above to
Please note the name difference for the Jetty related files. Here the location in the build file.
Please make sure to check in and deliver all the changes to the component Build of the stream you are working on. Create a baseline for the component Build. Replace the component Build in the other streams with the new component baseline.
Please also make sure to correct the name of the ECJ in the ANT arguments on the ANT Tab for all ANT Build definitions.
Now the builds should work again.
There is work under way to fix this and potentially ship a backport. See defect 515939.
I just published the Work Item Command Line (WCL) version 5.0. In this post, I will provide a short summary of the changes that went in.
Download and License
The work item command line is provided as is, with no warranty or support, under MIT license. The source code is provided here.
In addition to the source code I usually create releases that contain a zip file with WCL pre built and ready to use. Please note that some of the commands require additional libraries that are not packaged due to license concerns. Read the readme file contained in the WCL folder for how to get the libraries. This is the WCL 5.0 release.
Compatibility
Please note that the current WCL 5.0 is built and tested for RTC 6.0.x. It should be possible to use WCL against older versions of RTC, but that requires some modifications to deal with some small API and packaging changes for RTC 6.0.x. Please see this post for some information about what needs to be done.
A new command -bulkupdate has been added that allows to perform an update for all work items returned by a query. All qualified work items will be updated with the values provided. The values can be provided with the same syntax used in the command -update.
It is now possible to delete all attachments of a work item. Use the pseudo attribute @deleteAttachments with required value “yes“.
It is now possible to delete all links of a linktype from a work item. Use the pseudo attribute @deleteLinks_linktype with required value “yes“.
Timestamps/Date values can now be set to unassigned.
Work item resolution values are now correctly exported as display value and not as resolution ID.
Work item resolution values are now correctly imported. WCL detects the value to set for a resolution from the display name and the ID.
The command -importworkitems now handles empty column values for the attributes where it makes sense. An empty column value will overwrite an attribute value and remove it. In previous versions the import ignored empty values. Not all attributes can be set to an empty value. Categories, for example, have to always be set using a category value. Use the switch /ignoreemptycolumnvalues to switch back to the old behavior ignoring empty columns.
The command -importworkitems does no longer attempt to write attributes that can not be set and only displays a warning. Examples are the creator, the creation date and other values managed by RTC.
The command -importworkitems handles some new pseudo attributes such as e-signature gracefully by warning that they are not supported.
The export and print work item commands order the column output alphabetically if no special column order is specified.
WCL has come to a point where it does pretty much all I wanted it to do in the inception phase. I hope it helps users out there. Some feedback who uses it and against which version would be helpful.
I have answered many customer questions about the RTC Java APIs in the past. Many of the posts in this blog are the result of such questions. I have no privileged access to the RTC source code. As far as I can tell, there is no secret library of RTC API documentation that the IBM development team is hiding from everybody else. Even if there was such documentation, I do not have access to it. So, how do I come up with those answers?
In short, by using the available capabilities to search for answers.
The whole answer is, that it is not necessary to know everything. It is just necessary to know how and where to search for the answers. These days, there are basically two answers to where and how to search.
Search the internet and limit the search to sites that are likely to have relevant information. Examples are Jazz.net and stackoverflow.com. Jazz.net has the Forum and the development Wiki. Stackoverflow has a lot of questions and answers from many contributions.
I did contribute at stackoverflow as well, but gave up when they would not accept a link to external content as answer. The purpose of this blog is to be able to provide answers and share examples. For that very reason rsjazz.wordpress.com is also a good place to search. The search capability in the top right of this blog is also a valid place to search. The page Interesting Links is a collection of links to other sources I have come across over the years.
Very recently the Jazz community and Github have become a source as well.
Many questions in the context of RTC Extensions might be just related to Eclipse Plugin development. https://www.eclipse.org/ has a lot of examples and documentation around such questions.
Search the RTC SDK
If there are no ready examples and answers in the internet, there is a host of example code available in the RTC SDK. The RTC SDK contains at least the following:
Unit test code for RTC the Java API. These contain especially examples for the administration API, but also for other parts of the client API provided by the Plain Java Client Libraries.
The Java Code for the client API provided by the Plan Java Client Libraries, including the documentation that is used to create the JavaDoc for the Plain Java Client Libraries.
RTC Rich Client Platform code used in the RTC Eclipse Client, including comments. This code uses the public client API but also uses internal API.
RTC Client plug in code, including Client operation behavior such as advisors (pre-conditions), participants (follow up actions), aspect editors providing the Eclipse Admin UI and related information.
RTC Server API and RTC Server code, including documentation for interfaces and methods.
RTC Server plug in code, including server operation behavior such as advisors (pre-conditions), participants (follow up actions), aspect editors providing the Eclipse Admin UI and related information.
Client and server extension points and related code.
Code for asynchronous tasks.
All the code mentioned above is included in the RTC Server SDK that is shipped with RTC in the all download section. The RTC Client SDK that is shipped in addition since 6.0.3 only contains the client code which is kept compatible to earlier versions of the Eclipse client.
You can search the RTC SDK by using the capabilities provided by Eclipse. Especially the Eclipse Plugin Development (PDE) and Rich Client Platform (RCP) development tools provide a comprehensive set of features to search the RTC SDK. There are several approaches that can be used to search in Eclipse and the Eclipse PDE:
Java Search for types, interfaces, methods and other Java Related properties. This includes the capability to use search pattern e.g. use an asterisk. It is possible to specify the relationships to search for e.g. search for classes that implement an interface.
Search for Plugins, Extension points, references to extension points. This includes the capability to see the plugin.xml for other extensions, revealing all the implementation classes and relevant information that can be used again in 1 to narrow down the search to the relevant code.
Eclipse RCP/SWT Inspectors such as Yari or tools like the built in Plugin Spy allow to use the declarative and reflective capabilities of Eclipse and the RCP to analyze the UI and other information provided in the plugins and features. This allows to find which classes are called in menus and to look at which data is used by views and how it is used.
Other search capabilities such as File search and Text search can help in certain conditions.
These capabilities allow to pretty much find anything that is available in the RTC SDK. This does not make it trivial to understand what is found, especially the RCP UI code of RTC is sometimes very hard to understand, but it provides at least entry points, where to start.
Once something is found, there are various means available to
Open or navigate to the element that is found
Open the package containing the element, if applicable
Open other items that are related to the found element
This is really all one needs to know. The rest is just using these capabilities to your advantage. Some examples how to use this are given below.
Finding Built-in Extensions in the SDK
The SDK contains all the client and server extensions and their source code. When developing a custom advisor or follow up action e.g. following the Extensions Workshop it is a good idea to get inspiration from the existing code. How does that code work? How does it use the API?
Any such scenario starts with finding examples that use one of the relevant extension points. The extension points are shown in the Extensions Workshop and other examples. Most likely there is an example using an extension point in the plugin.xml already, or a new plugin.xml has been just created.
In the open open editor of the plugin.xml on the Extensions tab, right click the extension point that is of interest. To understand where this is used select Find References and click to start the search.
Search for references to an existing extension point.
The search window will open below and show the references found after a while. This can take some time, dependent on the performance of the client computer. The search result will show multiple hits. The image below shows a part of the hits in the current workspace. It especially shows the references from RTC SDK code to the extension point used by server operation advisors. All the references shown here have the same namespace prefix com.ibm.team. This hints that the origin of the code is in the RTC SDK.
Operation Advisors shipped with RTC.
Use the up and down arrows to browse the referencing plugin XML’s, or click one of the rows in the search that is of interest. Make yourself familiar with the name spaces that are used. This makes it easier to focus on the important examples you are looking for.
The namespaces refer to the following components of RTC
com.ibm.team.apt – Agile Planning
com.ibm.team.build – Build
com.ibm.team.filesystem – Source Control Filesystem
com.ibm.team.scm – Source Control
com.ibm.team.workitem – Work Items
When looking for an example for SCM related operations such as deliver, look at the examples in the namespaces filesystem and scm. When looking at work item related extensions look into the examples with the workitem namespace.
Click the search result to open the related plugin.xml. As an example the search result com.ibm.team.workitem.service. Eclipse allows to find strings in open files, so it is possible to search for strings using CTRL+F. E.g. search for Required to find because you are interested in advisors for required attributes.
Required Attributes by Tape and State Advisor
In the plugin.XML you can see the implementing class. You can use the full qualified class name e.g. com.ibm.team.workitem.common.internal.RequiredAttributesByTypeAndStateAdvisor to open the class using the Navigate>Open Type menu of Eclipse.
Open the implementation class of the advisor
This repository workspace has the SDK set up as well as the RTC Plain Java Client Libraries. Select the class in the SDK. Browse the classes code and documentation. Open the Class in the package explorer.
Explore the package containing the class
In the package explorer view in Eclipse browse the package e.g. C:\RTC605Dev\installs\rtc-server-sdk\plugins\com.ibm.team.workitem.common_3.2.900.v20170928_1641.jar.
Scroll down to the lower area. There are some packages that do not contain Java classes. For example there is a package or rather folder, that contain files such as the schemas used by the aspect editors in the admin UI. You can open the files to look at the details.
Package explorer shows related content.
Please note: the one I was really looking for is a bit elusive, and I have not been able to locate it until now. I will update the post as soon as I have a solution.
This is only one way of doing it. Dependent on what is available and what one knows there are many other ways to look into the source code. For example search for all extension points with a package name of com.ibm.team* to find all the extension points. Search for specific Plugins, in case the name is available.
Search for References to an extension point.
Try to explore different angles to find what you are looking for.
Search the Java Code
As already mentioned, it is possible to search for Java classes. This can also use search patter using an askerisk. Search for different aspects such as methods and interfaces and limit the search result set.
Once a class is open, it is easy to navigate within the class hierarchy, open the jar file in the package explorer etc..
As seen above, there are many ways to find stuff in the RTC SDK that can be used. As always I hope this helps users out there with their endeavors. I will try to add more examples.
