I am very passionate about the RTC Extensions Workshop as you might be able to tell from the content of this blog. Performing it with EWM 7.0.x provided several challenges. It became apparent that an update to the workshop would be beneficial.
I spent a considerable amount of time in the past two months to update the workshop. As a summary the following items where addressed:
Since the CCM server is shipped with WebSphere liberty profile, configuring the server for debugging needed to be changed. The old way to configure the server still worked in the 6.0.x versions, so it went unnoticed. With EWM 7.0.1 this is no longer the case and the workshop was updated to address this.
The advanced capabilities introduced in the EWM SCM system in the 6.x and later caused a deviation of the screen shots showing the pending changes. The workshop setup tool was slightly changed to fix this.
The workshop setup tool and its shell script has been tested with Linux and MAC OS.
I wanted to add a section to Lab 1, explaining how to setup the existing Eclipse client/server development workspaces to better support development and debugging of the Plain Java Client Libraries forever. The new last optional section addresses this. For this reason Lab 1 of the workshop is a must for anyone intending to create Java based automation or extensions to RTC/EWM.
I had an errata list with a number of small issues, typos, naming inconsistencies and the like that were fixed. During reviews a bunch more showed up and were fixed.
A colleague ran the workshop on his MAC, so this works. Use whatever is available for MAC like Eclipse and where this is not specifically available, use the Linux versions.
The RTC Extensions Workshop has been published with an additional section for the new EWM versions and is now available for download. I will update recent posts around the workshop in the next few days.
As always, I hope that this blog post helps the users in the Jazz Community.
As mitigation I suggested to replace the JRE with an earlier one, or one that works. Unfortunately, this does not look sustainable. I have tried downloading the newest IBM Java SDK and the issue still happens.
I have looked a little bit deeper into it and discovered that the server.startup script provides a debug option. Using this option allows remote debugging the EWM server. I am looking into an update to the RTC/EWM Extensions Workshop for 7.0.x. until this is available here a procedure how to follow the workshop for 7.0.x versions.
In Lab 1.2, do not add the following lines to the server.startup script, remove these lines:
set JAVA_OPTS=%JAVA_OPTS% -Xdebug
set JAVA_OPTS=%JAVA_OPTS% -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3388
Scroll to the bottom of the server.startup script and inspect the statements behind setting the JAVA_OPTS. Note that there are an action and an option supported.
Start server for remote debugging
The action -debug can be used to start the Liberty Profile Server in debug mode. When following the workshop, continue to Lab 1.5.
Follow the lab until 1.5__32 Add the following steps:
Open a console/cmd window and change directory into the server folder containing the server.startup script. Stop the development server by running server.shutdown.
Wait until the server is successfully stopped. Start the development server by running the following command.
Note that the server does not start, it suspends listening for a remote connection on a dt_socket/port. The default socket/port address is 8000.
The server will not continue starting unless a remote connection is opened on that port.
Connect Eclipse as remote debugger
Follow the workshop and open the [RTCExt] Debug Running Liberty Profile launch configuration. This launch is used to connect to a remote Java application. In our case the application is the RTS/EWM development server waiting on socket/port 8000.
Change the port in the launch to 8000 then click Apply and Debug.
Switch to the command prompt. It takes a while and the server continues to start. Wait for the server application to successfully start.
Continue with the workshop and remote debug the server.
When disconnecting from the remote Java application, the RTC development server will show again, that it waits for a remote connection.
Reprovision the Jazz applications
The option -clean can be used to force to re-provision the Jazz Applications installed on the Liberty Profile server. This is necessary to force rereading custom server extensions during deployment of these extensions. The option can also used together with the -debug action.
While running the test, I found that I had to increase the maximum heap size for the launch [RTCExt] Jetty RTC Server, because the JUnit Test that is run to create the test database, ran out of heap memory. I doubled the value from -Xmx256M to -Xmx512M and the error was gone.
The fix is only in the EWM/RTC Client SDK, so it is only necessary to download that again.
Please note that due to the name changes and newer versions of Eclipse, there might be some small differences between the screen shots in the RTC Extensions Workshop and the newer versions. However, they are so minor, it should not be an issue. I am currently looking into updating the current Extensions Workshop material with small changes to help guide the user with these changes.
I have seen issues with the RTC Client SDK for the 7.0.x releases recently. The Server SDK seems not to be affected.
Client plug-in development
UPDATE: The issue with the Client SDK has been found and corrected. Just download and use the new Client SDK.
UPDATE: The issue with the server debugging can be solved as explained below.
If you are using the Rational Team Concert Extensions Workshop for the EWM/RTC versions 7.0.0, 7.0.1, 7.0.2 and try to start an Eclipse client from the client launch as described in section 1.9 Test the RTC Eclipse client launch, you might run into a problem. I reported the issue in Defect 511733 (use your Jazz.net credentials).
This issue has been solved and you should not see it any longer. Make sure to download the latest Client SDK from Jazz.net.
If you need to develop client extensions and run into this problem, check the status of the defect. Worst case, use a 220.127.116.11 SDK to develop your extension. This is absolutely possible if you do not rely on a new client API feature from 7.0.x.
If you are using the Rational Team Concert Extensions Workshop for the EWM/RTC versions 7.0.2, trying to launch the development server in debug mode that allows to attach an Eclipse debugger to that server, the server startup fails. In ‘1.2__15 start the server using server.startup.bat’ after adding the debug configuration, the server does not start. The log folder only contains the files start.log and console.log. The content indicates that the connection to the debug port failed because the port was already in use. When checking the pot usage, the port is not in use. This has not been seen in any version before. See Defect 527867 for work arounds and solutions.
The SCMUtils is based on a framework that I developed over time. The framework provides easy to reuse mechanisms to create custom commands that handle most of the essential requirements automatically. It handles calling a command based on a name string, the parameters the command needs and optional parameters. The command can define its own help content that is printed when either the command was called with missing parameters or if the command name is not supported. This blog post explains the steps needed.
It is easy to implement your own commands. The framework project provides abstract classes that can be used to implement own commands.
The class SampleCommandCmd is a simple example that can be used as a starting point. It implements all the basic capabilities that are needed.
The class AbstractCommand can be extended to a simple command that can execute your own code.
The class AbstractTeamrepositoryCommand is an abstract class that can be used to implement custom commands that interact with a Jazz application. It manages the parameter and behavior required for connecting to a Jazz server. In addition it automatically implements scenario logging. It can be extended to implement commands that work against a Jazz server.
It is not necessary to use one of the classes above. It is only required to implement the interface ICommand.
Tips and tricks for implementing custom commands
The framework comes with a sample command, that you can look to understand how the implementation works. If you want to create your own command, you should not do that in the project com.ibm.js.team.supporttools.framework. You should add it in a project area that refers to the com.ibm.js.team.supporttools.framework project. E.g. you can add the command in the project com.ibm.js.team.supporttools.scmutils.
Open the command /com.ibm.js.team.supporttools.framework.SampleCommandCmd and study the class.
The class SampleCommandCmd extends AbstractCommand, so it does not inherit logging into a jazz server. It also implements ICommand (which it inherits from AbstractCommand).
The next part is to define a logger. The framework uses Simple Logging Facade for Java (SLF4J) to provide logging. The log file will be written to disc. Use the logger to add custom logging.
The command must implement the default constructor and pass the command id to the framework. The super class implements this, so the command name is passed to the super class.
In addCommandOptions(), the class adds additional options (parameters). The first parameter in addOption() is the option/parameter name, the second defines if the option requires a value. The third parameter is the description of the option.
The command has to make sure it gets the options it needs. This is checked in the next step. The method checkParameters() is used to do just this. Test if the required options are available.
The method printSyntax() is called if there is some kind of issue with calling the SCMUtils. Implement your documentation here. The logger by default prints log level info.
The entry point for the custom code is the execute() method. This usually gets all the mandatory options and their values. The method getCmd().getOptionValue() is used to get the parameter or option value if the option has a value. The method getCmd().hasOption() is usually if the option does not have a value and is more like a flag.
Then it usually executes the payload code that does whatever the command is supposed to do. The return value should be true if the command succeeded and false otherwise.
Adding the custom command to the CommandFactory
After implementing a new command, it has to be added to the ScmSupportToolsFactory in the project com.ibm.js.team.supporttools.scmutils to make it available to be called..
This is where the command needs to be added. See the available commands.
Remove the comments before the put(new SampleCommandCmd()), to enable the sample command. You would add a put for your own command here. The SampleCommand is a good learning execise if you want to learn debugging.
It is really simple to add new commands, there is a set of commands already available, copy one of those if it looks similar to what you want to do.
I have answered many customer questions about the RTC Java APIs in the past. Many of the posts in this blog are the result of such questions. I have no privileged access to the RTC source code. As far as I can tell, there is no secret library of RTC API documentation that the IBM development team is hiding from everybody else. Even if there was such documentation, I do not have access to it. So, how do I come up with those answers?
In short, by using the available capabilities to search for answers.
The whole answer is, that it is not necessary to know everything. It is just necessary to know how and where to search for the answers. These days, there are basically two answers to where and how to search.
Search the internet and limit the search to sites that are likely to have relevant information. Examples are Jazz.net and stackoverflow.com. Jazz.net has the Forum and the development Wiki. Stackoverflow has a lot of questions and answers from many contributions.
I did contribute at stackoverflow as well, but gave up when they would not accept a link to external content as answer. The purpose of this blog is to be able to provide answers and share examples. For that very reason rsjazz.wordpress.com is also a good place to search. The search capability in the top right of this blog is also a valid place to search. The page Interesting Links is a collection of links to other sources I have come across over the years.
Very recently the Jazz community and Github have become a source as well.
Many questions in the context of RTC Extensions might be just related to Eclipse Plugin development. https://www.eclipse.org/ has a lot of examples and documentation around such questions.
Search the RTC SDK
If there are no ready examples and answers in the internet, there is a host of example code available in the RTC SDK. The RTC SDK contains at least the following:
Unit test code for RTC the Java API. These contain especially examples for the administration API, but also for other parts of the client API provided by the Plain Java Client Libraries.
The Java Code for the client API provided by the Plan Java Client Libraries, including the documentation that is used to create the JavaDoc for the Plain Java Client Libraries.
RTC Rich Client Platform code used in the RTC Eclipse Client, including comments. This code uses the public client API but also uses internal API.
RTC Client plug in code, including Client operation behavior such as advisors (pre-conditions), participants (follow up actions), aspect editors providing the Eclipse Admin UI and related information.
RTC Server API and RTC Server code, including documentation for interfaces and methods.
RTC Server plug in code, including server operation behavior such as advisors (pre-conditions), participants (follow up actions), aspect editors providing the Eclipse Admin UI and related information.
Client and server extension points and related code.
Code for asynchronous tasks.
All the code mentioned above is included in the RTC Server SDK that is shipped with RTC in the all download section. The RTC Client SDK that is shipped in addition since 6.0.3 only contains the client code which is kept compatible to earlier versions of the Eclipse client.
You can search the RTC SDK by using the capabilities provided by Eclipse. Especially the Eclipse Plugin Development (PDE) and Rich Client Platform (RCP) development tools provide a comprehensive set of features to search the RTC SDK. There are several approaches that can be used to search in Eclipse and the Eclipse PDE:
Java Search for types, interfaces, methods and other Java Related properties. This includes the capability to use search pattern e.g. use an asterisk. It is possible to specify the relationships to search for e.g. search for classes that implement an interface.
Search for Plugins, Extension points, references to extension points. This includes the capability to see the plugin.xml for other extensions, revealing all the implementation classes and relevant information that can be used again in 1 to narrow down the search to the relevant code.
Eclipse RCP/SWT Inspectors such as Yari or tools like the built in Plugin Spy allow to use the declarative and reflective capabilities of Eclipse and the RCP to analyze the UI and other information provided in the plugins and features. This allows to find which classes are called in menus and to look at which data is used by views and how it is used.
Other search capabilities such as File search and Text search can help in certain conditions.
These capabilities allow to pretty much find anything that is available in the RTC SDK. This does not make it trivial to understand what is found, especially the RCP UI code of RTC is sometimes very hard to understand, but it provides at least entry points, where to start.
Once something is found, there are various means available to
Open or navigate to the element that is found
Open the package containing the element, if applicable
Open other items that are related to the found element
This is really all one needs to know. The rest is just using these capabilities to your advantage. Some examples how to use this are given below.
Finding Built-in Extensions in the SDK
The SDK contains all the client and server extensions and their source code. When developing a custom advisor or follow up action e.g. following the Extensions Workshop it is a good idea to get inspiration from the existing code. How does that code work? How does it use the API?
Any such scenario starts with finding examples that use one of the relevant extension points. The extension points are shown in the Extensions Workshop and other examples. Most likely there is an example using an extension point in the plugin.xml already, or a new plugin.xml has been just created.
In the open open editor of the plugin.xml on the Extensions tab, right click the extension point that is of interest. To understand where this is used select Find References and click to start the search.
The search window will open below and show the references found after a while. This can take some time, dependent on the performance of the client computer. The search result will show multiple hits. The image below shows a part of the hits in the current workspace. It especially shows the references from RTC SDK code to the extension point used by server operation advisors. All the references shown here have the same namespace prefix com.ibm.team. This hints that the origin of the code is in the RTC SDK.
Use the up and down arrows to browse the referencing plugin XML’s, or click one of the rows in the search that is of interest. Make yourself familiar with the name spaces that are used. This makes it easier to focus on the important examples you are looking for.
The namespaces refer to the following components of RTC
com.ibm.team.apt – Agile Planning
com.ibm.team.build – Build
com.ibm.team.filesystem – Source Control Filesystem
com.ibm.team.scm – Source Control
com.ibm.team.workitem – Work Items
When looking for an example for SCM related operations such as deliver, look at the examples in the namespaces filesystem and scm. When looking at work item related extensions look into the examples with the workitem namespace.
Click the search result to open the related plugin.xml. As an example the search result com.ibm.team.workitem.service. Eclipse allows to find strings in open files, so it is possible to search for strings using CTRL+F. E.g. search for Required to find because you are interested in advisors for required attributes.
In the plugin.XML you can see the implementing class. You can use the full qualified class name e.g. com.ibm.team.workitem.common.internal.RequiredAttributesByTypeAndStateAdvisor to open the class using the Navigate>Open Type menu of Eclipse.
This repository workspace has the SDK set up as well as the RTC Plain Java Client Libraries. Select the class in the SDK. Browse the classes code and documentation. Open the Class in the package explorer.
In the package explorer view in Eclipse browse the package e.g. C:\RTC605Dev\installs\rtc-server-sdk\plugins\com.ibm.team.workitem.common_3.2.900.v20170928_1641.jar.
Scroll down to the lower area. There are some packages that do not contain Java classes. For example there is a package or rather folder, that contain files such as the schemas used by the aspect editors in the admin UI. You can open the files to look at the details.
Please note: the one I was really looking for is a bit elusive, and I have not been able to locate it until now. I will update the post as soon as I have a solution.
This is only one way of doing it. Dependent on what is available and what one knows there are many other ways to look into the source code. For example search for all extension points with a package name of com.ibm.team* to find all the extension points. Search for specific Plugins, in case the name is available.
Try to explore different angles to find what you are looking for.
Search the Java Code
As already mentioned, it is possible to search for Java classes. This can also use search patter using an askerisk. Search for different aspects such as methods and interfaces and limit the search result set.
Once a class is open, it is easy to navigate within the class hierarchy, open the jar file in the package explorer etc..
There are several scenarios that could be of interest. Some of the scenarios will be provided below. A general documentation about how the authentication of a client to a Jazz application works is provided here in the development wiki.
Common scenarios are as follows:
Using a REST client to explore the REST/OSLC APIs
Creating custom automation using REST/OSLC APIs
Scenario 1 is also used in the OSLC Workshop. However, as explained below, things have changed since the workshop was initially written and the latest changes are not addressed in the workshop.
The main changes that could be recognized are in the browsers and the integration of REST clients. These changes heavily impact how easy or hard it is to login. The first REST client used in the OSLC workshop was the Firefox browser addon REST Client . Using this addon It is easy to login to the Jazz application. All that needs to be done is
Open a browser window
Provide the URL to the CLM application and open the URL
The application prompts for a login
Provide user name and password and perform the login
Open the REST Client Addon
The REST client addon uses the login performed in the steps above for subsequent calls to the server. This makes it easy, because it is transparent and there is no reason to worry about the details. The browser basically handled them.
Similar addons are available for the Chrome Browser. Examples are
The browser integrations above all allowed to use the browser to login and use the login information in subsequent calls. These browser addons or extensions used to be very popular, but security threats and changes in integration philosophies seem to make them more and more unavailable. Some of the addons mentioned above are discontinued and might not be available much longer.
The Postman as packaged app as explained at the end of this post is currently still available and allows to configure a Postman Interceptor, that intercepts the login information from the browser and uses it in later calls. So it works like the solutions 1-3 above and the REST clients available as Firefox addons or extensions.
Alternatives are standalone solutions like Postman as standalone App which seems to be the path forward for Postman. There are many other stand alone REST clients. The caveat is, that the simple option to just login to the Jazz application is no longer available as an option. So it is important to know the details,
When trying to write custom tooling that needs to access data on a CLM/ELM server, it is also necessary to understand the details.
Testing and special considerations
It is a good idea to have a test system. Ideally local on your own machine. You can follow Lab 1 of the Rational Team Concert Extensions Workshop to set up a small system with JTS and RTC. It is not necessary to install the plain Java client libraries, any of the SDKs or to run the WorkshopSetup application. It is possible to install the other applications as well sing the Web Installer or one of the many Installation Manager repositories.
Create the sample project in the lifecycle project administration section to have data to play with.
This blog shows the examples using Postman as standalone App which is very useful and allows to save and organize calls as a benefit. Like any application there are usually settings that can be beneficial or get into the way. Here the settings of interest in the context of this blog.
Disable SSL Certificate Verification
Please be aware that the SSL certificates of a test system as described above will typically be considered invalid. It is very common that test systems have invalid SSL certificates. A common issue is that the certificate is self signed and often for a different or generic host name. In such cases SSL certificate validation throws an error. Browsers, REST clients and APIs often reject working with servers presenting invalid SSL certificates. It is usually necessary to create exceptions or to configure the systems to ignore the SSL validation error.
Make sure the REST client or API works with sites that have invalid SSL certificates, or provide valid SSL certificates.
Postman rejects communicating with servers using invalid SSL certificates and displays an error message. This can be changed by changing the settings, turning the setting ‘SSL certificate verification’ to off. Postman now allows communicating to servers using invalid SSL certificates.
As an alternative, it is possible to provide valid certificates. It is also possible to use any other test system that might be available, especially when dedicated test systems with valid certificates are available. The screen shots in this blog refer to a test system that has been set up similar to the Extensions workshop, but with all applications installed and the sample lifecycle project being deployed.
When following redirects automatically, postman also collects several cookies. To focus on the really important details, and understand which headers and cookies are really needed, the setting below is very helpful.
The easiest to use method that has worked for me so far has been the Form Based authentication. This works in a simple set up of CLM with any application server in a standard configuration without Jazz Authorization Server (OIDC) installed.
Perform a Form based authentication login to an application
To perform the Form based authentication to the application use the context root of the JTS or the application and append /j_security_check to the context root to create the login URI.
The image below shows the simplest way of performing the authentication using Postman.
The POST to the secure resource URI provides the username and password using the form encoded key and value pairs for the j_username and j_password. password and username are both identical myadmin.
The code for this request is shown below.
POST /jts/j_security_check HTTP/1.1
Please note that there are no additional headers necessary. The request does not specify anything in the request for Parameters, Authorization, Headers, Pre-request Script or Tests.
The response shows status 302. There is no header X-com-ibm-team-repository-web-auth-msg. There is no header Location with any value, let alone /auth/authfailed. This indicates the authentication was successful.
The Cookies keeps the LtpaToken to authenticate.
If the setting for Automatically follows redirects is on, and other header options are also set to on, the request would be redirected and likely show a status 200. There would be a lot more headers and Cookies collected as well.
Once successfully authenticated, it is possible to send other requests passing the Cookies. For this OSLC request it is also necessary to pass an accept header and the OCLS-Core-version header.
Please note that the Jazz Product APIs might need other special handling. In the above example the communication returns an additional Cookie JSESSIONID. In subsequent calls provide a header JSESSIONID with the value of the returned Cookie.
The easiest way to ‘log out’ is to delete the Cookies for the server.
The request above can also be created by using a special content header Content-Type:application/x-www-form-urlencoded and a raw post body with a special encoding.
The images below show how this looks like in Postman.
The Following image shows an unsuccessful authentication due to a typo in the password. This is only one of the possible flows, but the other scenarios are handled the same way.
The response shows status 302. There is however a header Location with value /auth/authfailed. This indicates the authentication was not successful.
Where to Authenticate
My understanding so far has been that the CCM and the QM server have their own authentication. The RM server delegates authentication to the JTS. The Lyo implementation uses the public URI of the applications to create the authentication request. For the RM application it uses JTS instead.
As a summary, for Basic authentication, a special header Autorization that contains username and password encrypted in Base64 is sent with each request. The request below shows the header
Authorization: Basic c29tZXVzZXI6c29tZXBhc3N3b3Jk
Testing and special considerations
When using Basic Authentication in Postman it is useful to have the Automatically follow redirects setting to be on.
To configure Basic authentication in Postman, select the Authorization type Basic Auth and enter the username and the password. Postman generates the authentication header. When using the API or other tools that do not create the authentication header, it has to be created and added.
The call below gets the workitems catalog for a RTC server on Jazz.net. This requires sending a valid Accept header and the OCLS-Core-version header.
When sending the request, the final response should have a status 200.
Please be aware that basic auth does not have a session handling and you have to always send the authentication header.
I am currently working with a customer trying to authenticate to one of their CLM systems. This seems to work except with the CCM server. When they try to authenticate they get the following message.
You have followed a direct link to log in to a Jazz server. This page has been presented to ensure that a malicious website cannot use cleverly crafted content to circumvent security. Please log in if you would like to access the server.
We are looking into what could cause this issue. The web does not provide any good explanation. I found some questions in the Forum and other places and think I noticed a pattern. Several requests I have seen used a wrong URI to authenticate.
It is currently unclear what is happening and how to resolve the issue the customer is seeing. I will update the post, if we can figure out what is going wrong.
This blog post tries to shed some light on how to log into a CLM System to be able to use the REST and OSLC APIs. This should work for manual work as well as for automation tools. This blog uses Postman, but any REST client should work. The details, especially covering redirects and different authentication methods can be found inn this document Authentication of a native client with a Jazz-based application.
As always, I hope that this post helps users out there with using their Jazz tools.
There is no such thing as limitless computing power. This is an unfortunate truth that can cause problems running the CLM and other tools, as the usage grows. To understand what systems actually do when getting under heavy load, more and more monitoring was introduced over the last years. Resource intensive scenarios where identified and the CLM tools have capabilities to record information about their frequency and duration. Plan loading and SCM compare workspace are examples in the product.
Custom Resource Intensive Scenarios
In addition to resource intensive scenarios that are built in, it is also possible to introduce custom resource intensive scenarios. Some examples are:
Custom automation that execute long running operations on work items, SCM data, requirements, test artifacts. Typical scenarios are custom export/import, mass updates, custom analysis of source code, baselines, linked work items.
Follow up actions
Long running custom dashboards
What is your server up to?
When users complain about performance problems, even if a server is getting overloaded, it is hard to find the root causes, because a typical server does so many things.
Monitoring that has been added over time has helped, but it is still hard. It is sometimes even hard to understand the situation. As an example for how complex this can become. Users complained about performance.
Our performance architect looked at the server load and the build load and a huge amount of calls that we were not able to account for. The server was unarguably under heavy load created by builds, but the build users and SCM users where not complaining. The developers we talked to had no real issues. Some users, at a different location, using work items and running work item queries, had.
Because we could not explain the inconsistent feedback, I finally went to the location where the users where complaining. I met the users followed their day to day work and found the work item performance unacceptable. The web browser was even locking up on them.
Knowing this, we were able to reproduce the use case, and look into what happened. We found that the work item load was slow, especially on slow laptops, because it had to load so many team areas and iterations. This was specific to how the project area was configured and used.
We also found that the browser flooded the server with requests that where definitely not part of what the product UI sent. This basically forced the Web Browser to process and cache thousands of calls, reserving more and more memory and exhausting the CPU capabilities of the relatively weak laptops used by the users that complained.
The final verdict was, that there was a custom extension to the theme that created all these calls. It took us weeks and was luck that we found this out. If we had known there was such an extension, we would have been able to find this a lot faster. The server was still under a heavy build load, but the performance issue reported was not related to that.
Needless to say that this extension was also deployed in other environments. If it had a detrimental impact, it was heavily depended on the timeline and iteration structure of a project area. The more and deeper the worse.
It would have helped if we could have seen the extension working, and see how long it worked would also have helped.
Registering Custom Resource Intensive Scenarios
The same mechanism that is used to register resource intensive scenarios in the product code can be used to register custom resource intensive scenarios. Unfortunately, we where lacking a good description and supporting code that we could provide customers to use it for their extensions.
This has now changed. Some colleagues and I, independently, started creating a customer usable description how to register resource intensive scenarios. A colleague wrote some cURL code to do this. I wrote Java code to do this and started creating a presentation. When we found out, we decided to combine the effort. Here the result.
There is basically a REST API to register the start and the stop of a scenario. All there is to register the start of the scenario at the beginning and then register the stop, after you are done. See Register Custom Scripts as a Resource Intensive Scenario for more details on the code.
What should your automation do?
If you have written automation tools or extensions, you should use the methods described in Register Custom Scripts as a Resource Intensive Scenario, to register your extension as an resource intensive scenario. Add the code to register the start and stop in a way that allows for disabling it easily.
Monitor the various resource intensive scenarios over time. For a scenario that takes only a fraction of a second, you could temporarily disable the registration. Scenarios that take a second or longer should continue to be monitored.
If you have questions around the Custom Resource Intensive Scenario code, ask them in the Jazz.net forum instead of commenting on the article or this blog post. Tag the question as a clm question and add the tag: custom-resource-intensive-scenarios to mark it for the reader.
Please use the method above to enhance your automation and extensions to allow monitoring their duration, frequency and deviation.
As always I hope this helps users out there with the Jazz products.
I worked with a colleague on the Work Item Command Line who was adding a feature for usage with a customer. That colleague had also done some changes related to availability and performance running queries. These changes where merged back into the code. WCL was missing some capabilities that I always wanted to have. I had a quick look to understand how much effort hat would be and took the opportunity to implement these capabilities on the way. While doing so, some small enhancements where added and some defects where found and fixed. All that needed to be published as a new version of WCL.
Readme documents for deployment in source code updated
Refactored the capability to create a string representation for a work item attribute, link or pseudo attribute (e.g. attachments) into a helper class WorkItemExportHelper
Migrated -exportworkitems to use the new helper class WorkItemExportHelper
Several fixes in -exportworkitems to
Enable error handling and make suppressing errors work
Introduce a new flag that exports all supported columns and links
If exporting all columns, order the attributes and links (except ID, Type and Summary that are by default at the beginning)
Checked functionality with newest OpenCSV
Refactoring class to make maintenance easier
Added a new command -printworkitem that prints the attributes, attachments and supported links of a work item
Using the WorkItemExportHelper and its capabilities
Also provides switch to print all attributes and links ordered as above
Exports the attachments to a provided folder
Added a new command -printtypeattributes that allows to get the attribute Id’s and Display Names for a given work item type in a project area
Added a command -validateoslclinks that validates OSLC links and their back links in other applications fixing missing back links
External library dependencies are added without a version number to allow newer versions
Please keep in mind that the information in this blog is “as is”, unsupported, and may be outdated or inaccurate.
The Work Item Command Line (WCL) is not an officially supported application.
For information on released products, consult the product documentation, support tech notes, and the Jazz.net library.
Some of the API used in the Work Item Command Line becomes deprecated.
Some changes especially for -importworkitems needed for versions of RTC e.g. 6.0.5 and later
These will likely not work with earlier versions of RTC
if you need that either try an earlier version of WCL such as 4.0 or 4.1 or open an issue and I could look into a backport
The Linking API in RTC has evolved and deprecates some API used in WCL
OpenCSV has evolved and newer versions deprecates some API used in WCL
OpenCSV requires a new library commons-lang3-3.1.jar
Migration to Maven might be something to consider
This is all not problematic yet, but needs to be addressed sooner or later.
WCL is released under the MIT License. See the License.txt and the license headers in the individual files.
This code has been used with RTC 4.x, 5.x and 6.x with no or minimal changes and it is pretty safe to assume, that the code will work with newer versions of RTC to come.
An exception is the -importworkitems command that requires a library to use a mapping file. The capability to use the mapping file was moved into the plain Java Client libraries in RTC 6.0.5 as an internal class. To support work item import with the mapping moved to the Plain Java Client Libraries as internal API, the imports in ImportWorkItemsCommand where changed and the file com.ibm.team.workitem.rcp.core*.jar was removed from the jar file dependencies in the build path.
See Restore Compatibility to 6.0.4 and before for how to switch this back.
The code requires the RTC Plain Java Client Libraries.
The Export and import commands require additional external libraries that need to be downloaded and installed separately. See the document below.
Restore Compatibility to 6.0.4 and older
To support 6.0.5 and beyond the imports in ImportWorkItemsCommand where changed from:
and the file com.ibm.team.workitem.rcp.core*.jar was removed from the jar file dependencies in the build path.
Follow the description in the sections beginning with Downloads how to get and rebuild the code. The steps in this section below are required to port WCL back to RTC 4.0.4 and earlier:
The original imports have been put back in past 4.0.3, but commented out. To enable the mapping for earlier versions of RTC, replace the imports *BugzillaMapping* with the versions commented out. Then add the com.ibm.team.workitem.rcp.core_*.jar file back in to the build path and add the required JAR file as below
The import commands of WCL can only provide the capability to use a mapping file by using a JAR file that only ships with the RTC Eclipse client and the SDK. The classes used for the mapping file capability are located in the library com.ibm.team.workitem.rcp.core. Open the Install location of the RTC Eclipse client and search for com.ibm.team.workitem.rcp.core*. You should find a file names similar to this one: com.ibm.team.workitem.rcp.core_3.1.900.v20141010_0124.jar. The version numbers at the end could be different. Copy the JAR file into the into the libfolder of your version of WCL.
Use the option Download ZIP to download the code as ZIP file. Once the download finishes, browse the folder structure of the ZIP file. The top level contains the README.md file. An Eclipse project is contained in the folder com.ibm.js.team.workitem.commandline. This project can be used to develop the code and to generate an executable.
It is also possible to clone the repository and import the project using GIT. Clone the GIT project to your local file system. This requires GIT or some GIT GUI to be installed. How to use GIT is out of the scope of this blog. See the help how to set up GIT and how to use it with Eclipse.
It is also possible to download releases from the releases page. The zip code made available is in the same format as the one above can be used as described above. Note that the releases might be older, and there might be no release available for the latest code version.
See below how to import the enclosed project into Eclipse.
Some releases might contain a prepackaged executable for convenience e.g. named wcl-V4.3-20190117.zip. See the section Deploying the packaged application below for how to use this form of packaging. Please note that the prepackaged executable might not be available for all versions. The following sections explain how to create this executable. See the section Packaging the application for shipping for how to perform the packaging.
Install a current Eclipse or RTC Eclipse client.
Install a current Java 1.8 JRE or Java 1.8 JDK set it as active in Eclipse as Installed JRE and Execution Environment.
Getting the code, install and deploy
Once you have installed such an environment do the following
Import the Code
Open the Java Perspective in Eclipse.
Import the project into Eclipse from the file system.
To import the ZIP file use File>Import, select Existing Projects into Workspace. Click Select archive file, browse to and select the ZIP file you downloaded. Import the enclosed Eclipse project com.ibm.js.team.workitem.commandline.
To import the GIT repository use the GIT perspective to add the local cloned repository and then import the project com.ibm.js.team.workitem.commandline.
Create a User Library for the Plain Java Client Libraries
Use Window>Preferences>Java>BuildPath>User Libraries and add a user library named PlainJavaApi .
Add the JAR files of the Plain Java Client Libraries for the version of RTC you are using.
Rename the jar file e.g. commons-lang3-3.1.jar to commons-lang.jar
Copy the library file commons-lang.jar into /com.ibm.js.team.workitem.commandline/lib to be able to run from Eclipse
Check the project and recompile
Select Project>Clean and clean the project that was just imported.
There should be no compiler errors or issues with the classpath
The project should now have built and compiled and show no errors.
Building the WCL or extract from a Release
Follow the description in the file ReadMe – HowToRelease.txt in the root folder of the project
To create a wcl.jar file
To crate a WCL/lib folder
To provide batch and license files
Follow the description in the file /lib/ReadMe.txt in the lib folder of the project
To provide the required libraries in the WCL/lib folder
Select the folder, for example C:\Temp\wcl\ and compress the file
Rename the archive file to wcl-Vx-YYYYMMDD.zip,
where YYYY is the year, MM is the month and DD is the day
The file is now ready for publishing
Use the file wcl-Vx-YYYYMMDD.zip for shipping similar to the next section.
Deploying the packaged application
Assume a file was created following the steps above and the file is called wcl-V4.3-20190117.zip. This binary contains WCL as JAR file and other information such as licenses and Readme files. It also contains script files to set the environment and run the Jar File.
The folder lacks libraries needed to execute that need to be provided.
To deploy from this file, create a new folder as deploy location for the automation prototype. In the description here we use C:\temp. It is possible to use any other folder. In this case replace the folder in the this description by the folder you chose.
Extract the binary wcl-V4.3-20190117.zip file into the location C:\temp. There should now be a folder C:\temp\WCL. The folder among others contains the files wcl.jar, wcl.bat, wcl.sh.
Get the libraries as described in Download and Install openCSV and Download and Install Apache commons-lang above, and rename them as described in /lib/ReadMe.txt.
Check the script file and make sure to provide a valid path to a folder containing the unzipped Plain Java Client Libraries and a recent Java 1.8 is available over a Java_Home.
Call WCL using the script files or call it as Jar File as described in said script files.
Since some time now I started to run into a blocking issue with the Extensions workshop. I was not able to find a solution so far. Today a colleague saw the same and asked for help. Here what I found.
Problem: the browser prevents connecting to the Jetty server
When launching the Jetty Debug server, everything seems to be fine. You can even connect with an Eclipse client to the server. However, if you try to follow the description in the Extensions Workshop and you try to connect with a browser to https://localhost:7443/jazz/admin you see a screen like the one below.
In Chrome the error claims
This site can’t provide a secure connection localhost sent an invalid response.
Try running Windows Network Diagnostics.
Firefox is even more alarming and says
Secure Connection Failed
An error occurred during a connection to localhost:7443. Peer reports it experienced an internal error. Error code: SSL_ERROR_INTERNAL_ERROR_ALERT
The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.
Report errors like this to help Mozilla identify and block malicious sites
Can’t connect securely to this page
This might be because the site uses outdated or unsafe TLS security settings. If this keeps happening, try contacting the website’s owner.
Go back to the last page
It looks like the usual SSL issue with the standard certificate, but it actually is not. It does not allow you to proceed and add an exception to connect to the web site.
You are basically stuck.
I had the same issue yesterday with 6.0.5 and 6.0.6 and I knew it was working very recently – when I moved to my new laptop. I searched the internet and was not really sure about a solution still. The answers pointed at version issues and certification mismatches. So I decided to switch the Eclipse environment to a current JDK and that solved the problem. I have tried it with RTC 6.0.5 and 6.0.6 but I am pretty sure it would work with other versions as well.
Please note that the Extensions workshop suggests to use the same JDK the Server uses. I think since Browsers have recently increased their security measures, this is no longer true.
Here how to fix the problem.
Download and install a recent Java JDK/JRE that is compatible with the version of the RTC SDK you are working with.
6.0.6 is compatible with Java 1.8
I use a JDK because there are other development tools that require a JDK
Open the Eclipse server development workspace
Type JRE and navigate to Java>Installed JREs
Add the new JDK/JRE
Select the JDK/JRE as active click Apply and Close
Navigate to Java>Installed JREs>Execution Environments
Select the Execution Environment with the matching version
Select the new JDK/JRE as active click Apply and Close
Launch the Jetty debug server
Now try to login to https://localhost:7443/jazz/admin again. You still get an error, however the browser provides the Advanced link to continue connection to the site.
The different browsers display it different, but the concept is the same for all of them.
Recent increases of browser security checks and invalidation of certificates requires to update to recent versions of the Java JDK/JRE to be used with the Extensions workshop. As always I hope that this helps people out there.