EWM Attribute Customization Introduction

While working on Debugging EWM Work Item Attribute Customization JavaScript I thought about all the questions I have seen being asked about Attribute Customization on the Jazz.net forum. There are some common questions I have seen a lot. There are some common things users would like to do and there are some limitations that might get into the way. I thought, I would create a blog that summarizes what I know about attribute customization and also provide the important links. I also want to provide a short summary about, for all I know, what can and can not be done with Attribute Customization and the provided API. I already create something like that in RTC Process Customization – What you can and cannot do. I will follow its structure and reuse some content but will try to be more specific in this blog. This blog will be limited to the work item attribute customization, with some few exceptions. The blog post will mention possible solutions that go beyond attribute customization, where this seems reasonable.

Important links

The following links are the most important references about EWM work item configuration and attribute customization.

About Jazz.net forum questions

A lot of the input that I take under consideration comes from questions and answers on the Jazz.net forum. Some of the examples originate from customer requests.

Questions on Jazz.net and even customer requests do often not provide good requirements. They often provide some implementation idea instead. The real problem or business case is usually hidden and not exposed. So it is important to question the request to drill down and expose why the question was asked as it is and what the background of the question is. If the real reason for the question is exposed, the answer can be completely different.

One example is: “Can I auto subscribe users to a work item?”. In pretty much all cases I can remember, the purpose of this was to send an e-mail notification to a user e.g. when the work item was created. Adding a user to the subscribers might or might not be a viable solution, but the example clearly exposes the problem.

So take the questions on Jazz.net with a grain on salt and try to dig deeper to understand what the question is really about.

What is Process Configuration and Attribute Customization?

EWM allows to configure the process of a project area. There are several options that can be used.

  1. Work items, work item attributes, editor presentations, workflows can be configured.
  2. Roles can be configured
  3. Permissions can be configured
  4. Operational behavior can be configured using out of the box preconditions and follow up actions.

Attribute Customization is a work item related capability provided by EWM. It provides built-in capabilities that can be configured and thus be considered process configuration as mentioned above.

The capability types available for configuration are:

  1. Default value provider for built in attribute types.
  2. Calculated value provider to support calculation with work item attributes to support certain processes.
  3. Value Sets to provide customized sets of selectable values for work item attributes. Built in value sets allow to extract string values from an XML page in the Web, select contributors by role and allow to select enumeration vales based on other dependent enumerations.
  4. Validators to validate values of attributes based on range or regular expressions. Validators can show a warning in the UI and be used with a built in advisor to prevent saving a work item, unless the offending attribute values are corrected.
  5. Conditions can be used with built in advisors to make work item attributes read-only or required. There is no built in condition available at this time.

What is special about Attribute Customization?

What sets attribute customization apart from the other capabilities is, that it provides a JavaScript API that allows to create custom scripts for provider configurations. The JavaScript can be uploaded to the project areas process by any user with the required permission. This does not require write access to the application servers file system. For this reason it is possible to create and deploy custom JavaScript scripts even if the server is maintained by a cloud provider.

JavaScript is also reasonably easy to learn and a lot of practitioners have some JavaScript skills. This is a two edged sword.

Contrary to Eclipse extension plugins, another extension mechanism available in EWM, the JavaScript API has a relatively low experience threshold for beginners. It is way easier to develop than the other Eclipse plugin based customization options available in EWM that require great skills, understanding of the framework and familiarity with the available API. Since these options are based on Java and Eclipse plugin extension development, they require the plugins to be deployed which means they need to be copied into a file system, typically on the server where the CCM application is deployed. This is usually not possible in a cloud hosted environment. Except, some cloud services would perform such deployments for the customer, for a higher price tier.

The danger is that the low skill cap attracts very inexperienced users, which can eventually lead to issues, including server performance issues. Some of the capabilities can be overused. There is also a tendency to overcomplicate the work item workflow and the process.

The JavaScript Attribute Customization API constraints and limitations

Attribute Customization is configured on the attribute level. This means that any attribute customization will apply to all work item types that have the attribute configured.

JavaScript attribute customization scripts have to implement the required interface for their type. All the interfaces require to return some data. The data can be empty or undefined, dependent on the attribute type, e.g. an empty string or null. It has to return something that matches the attribute type it is configured for, nevertheless.

It is possible to detect the type of the work item the script is executed for, this allows to have different behavior for different work item types in the JavaScript attribute customization script. Based on the work item type attribute value, the script decides what to return, some custom value or the current attribute value to do nothing. This strategy allows to adjust for type specific attribute customization to some extend.

The API description in the Work Items Attribute Customization Wiki page explains which work item attribute value types are supported. Only very basic attribute types such as strings, numbers, enumerations, timestamps and dates are really supported. Especially note that the support for complex item based attribute types such as team areas, contributors etc. is very limited. For a those attributes, all you can access is the UUID and the display name. If you want to set such values, you must return the UUID of the item.

The documented API does not provide with methods to discover information beyond the work items current attribute values. There is no documented simple JavaScript API to discover details about not attribute based information. There is no documented simple API to follow or create links to work items or other artifacts. There is no documented API to access process information, such as users and roles or any other details. There is also no mechanism to detect a state change or workflow action. The workflow action is only available in conditions but the detection of a state change would be desirable in calculated values as well.

The return value of JavaScript attribute customization scripts is processed in the API. Dependent on the work item attribute type, the script is configured for, some kind of conversion is performed. As an example a string can be converted into a timestamp, provided the format is correct.

As far as I can tell, the conversion that is performed for html and string based attributes, including the subject and description, when returned from JavaScript converts the content into a string escaping any special content such as XML or HTML tags. This conversion happens outside of the JavaScript and there is no way to change the behavior. Because of this reason it seems to be impossible to return HTML with working links for HTML or string based attributes including the summary and description. Only the built in configurable default values work correctly. The only current attribute type that can be set by JavaScript attribute customization to contain links is the type Wiki. In this case the Wiki syntax has no XML/HTML tags involved that would have to be escaped.

Performance considerations

Some users seen to think there is such a thing as limitless size and performance. This does not apply to the real world. It is possible to heavily impact the performance of clients or servers with process configuration and especially customization. I am aware of a case, where JavaScript added to a Theme caused so much communication that the browsers on smaller laptops where drained of resources and almost came to a halt. Loading a work item took several minutes. This made working with the tools impossible for many users. So be careful and consider what you might do to the performance.

It is also very easy to annoy the users with overcomplicated processes, thousands of choices, required attributes nobody knows the values for etc. In general big numbers are bad and small numbers are good. When drop down boxes grow too big and the number of items ever increases, the users will be affected in their ability to successfully use the process. I have seen cases where the users got confused and did not realize that they had to enter a value and complained the UI had stopped. Overcomplicated processes will likely cause performance complaints and wear away the will to support the chosen tool and process.

There are limitations in the amount of custom attributes for a work item type (100). Having a lot of work item types is also not helping to make the process easier. Keep numbers low e.g. 10 would be a guidance for work item types.

Also keep in mind that there are size limits for all work item attribute types. Especially note that for list type attributes there is a size limit. How many selections can be stored is limited by the site of the list attributes and the size of the ID’s or item UUID’s that are stored.

Where does the JavaScript run?

The JavaScript runs in the Web Browser or in the EWM Eclipse client in a JavaScript interpreter..

Where and when is Attribute Customization not executing

Attribute customization including the JavaScript is not executed when the work item operations are created using an API such as OSLC or the SDK or Plain Java Client Libraries.

Work Item Proxy

When debugging the JavaScript, it is possible to see more objects accessible to the JavaScript in the debugger. It is possible to see an object called work Item proxy that looks very promising. Please note that this object is considered internal API. It should not be used by the JavaScript. The supported API is documented here. It is also important to understand, that this work item proxy is only available in the Web Browser and not if the script runs in the Eclipse client.

There are several posts on the Jazz.net that discuss how the proxy could be used. Again, it is internal API and should not be used, if you do, you are on your own. There might be unwanted side effects and the internal API could change at any time.

Why does my script not work?

This question comes up often on the forum. It is pretty much impossible to debug others scripts on Jazz.net. There are reasons for that:

  1. Java Script Attribute Customization depends on the process used in the project area. The community does not have and, due to privacy, security and other concerns will not get this.
  2. It is quite some effort to setup a test project area, even if one would have a process template.

So it is usually impossible to help on this level. What can be done is scanning the script and the description in the question for typical issues.

See Debugging EWM Work Item Attribute Customization JavaScript for how you can debug your own scripts.

Please note, there are different levels of issues with JavaScript. On the most fundamental level:

  1. Attachment Scripts must be enabled in the advanced teamserver.properties to be working – see below.
  2. The script must be accessible in the process.
  3. The script files encoding must be UTF-8 and consistent so the file can be read, interpreted and executed.
Process Attachment Scripts enabled.

If an error happens at this level the best you can hope for is an entry in the log file. Only if the JavaScript is successfully decoded and loaded, it can be debugged.

What you can and cannot do with JavaScript Attribute Customization

There are certain limitations in the JavaScript Attribute Customization API as far as I am aware. I provided a more comprehensive presentation here: Process Customization – What you can and cannot do.

I will try to provide some details here.

  1. Attribute Customization JavaScript is always defined as a function. Regardless what type of Attribute Customization the script must return a value (can be null or empty). Otherwise this will be a potential error in the log. I am not sure what happens worst case.
  2. Attribute customization is for the attribute. This means especially that the customization will be executed in all work item types that have the attribute configured.
  3. Sure, in JavaScript, you can get the work item type, by reading the respective attribute. You can then decide what to return. As explained in 1. you cannot return nothing. You however can get the current value of the attribute and return that, this helps in some cases.
  4. As far as I am aware, Attribute Customization only runs in the context of a users browser and the Eclipse client. As explained above, this also means that automation based on attribute customization does not work, if you access work items using any of the APIs. This is important, as pre-conditions/advisors and follow up actions/participants can be implemented as server extensions that always run, regardless who accesses the work item how. When the JavaScript is run in the Eclipse client, only the infrastructure of the Eclipse client is available.
  5. Attribute customization scripts can run multiple times, for example the calculated values might execute multiple times, based on changes to the other attributes they depend on.
  6. JavaScript Attribute Customization only has access to the data of the work item in which context it runs. It is not possible to follow work item links using the documented API.
  7. Even the available data of a work item is limited. As an example, a workflow action for a state change is only available in conditions. It is not available in any other attribute customization types, especially not in calculated values. This essentially means, there is no simple way of tracking and detecting a workflow state change.

Java Eclipse plugin extension based Attribute Customization

Beyond JavaScript, it is possible to provide Java based Work Item Attribute Customization Plugin Extensions. This is possible for all available attribute customization types, default values, calculated values, value sets, conditions and validators as shown in these examples. These Java based plugin extensions have access to more complex EWM Java SDK API. This makes it more feasible to create complex custom attribute customization. It is also easier to control which return result types are supported and, based on these, create richer attribute customization. It would, for example, absolutely be possible to have a default value provider that returns an XML or HTML result, that shows working HTML links. The decision to return a result that escapes the HTML tags or return the correct format is an implementation detail. A more complex example with additional background information can be found here and in this post.

The Java based attribute customizations would show up in the Admin UI in the same way the built in Attribute customization does. To make this possible the Java based Eclipse plugins need to be deployed in the Eclipse client as well as the EWM Server (see here). The API that is available to the Eclipse client and the server is the EWM Common SDK API. Consider reading this blog post to better understand what this means. The requirement to deploy these extensions on the EWM server are a potential inhibitor for cloud based deployments.

EWM Eclipse Plugin Extensibility

Java base attribute customization is only a subset of the Java Eclipse plugin extensibility available in EWM. The SDK provided for EWM allows to develop all kinds of extensions, including pre-conditions, follow up actions and asynchronous tasks. It is possible to develop such extensions to run in the context of the Eclipse client or on the server. These extensions need to be deployed in the context they are developed for. Server extensions have access to the full capabilities of the EWM Common SDK API and the EWM Server SDK API.

Preconditions also called Advisors can act as process advisors that prevent saving if the data violates some standards. Preconditions can be used to implement attribute customization validators or conditions. Example Advisors can be found here.

Follow up actions, also called Participants can perform additional operations on the EWM data. As an example, follow up actions can collect data across linked artefacts, perform calculations on the data, set attributes and properties for items and save the changes. Example participants can be found here. This includes the capability to create new work items and link the new work items to other work items.

In some cases it is better to use these capabilities instead of attribute customization, because it provides more API than even the Java based attribute customization.

All plugin based solutions require to be deployed. Server extensions require to be deployed in the server, which might be a blocker in an environment hosted as a service in the cloud.

Popular requests

In the next sections, I will try to provide some examples of popular requests. I will describe the background these request have, as far as I can tell, and which implementation options might be viable.

Popular: Custom E-Mail Notification

This comes up in various flavors. Examples are

  1. Can I automatically set an owner of a work item?
  2. Can I automatically subscribe a user to a work item?

With respect to attribute customization, the answer would be yes, you can do this, to some extent. It is possible to set or add a subscriber, to set or add a work item owner. In the JavaScript API this needs to be done the contributors UUID. The UUID will have to be configured or hard coded when using the JavaScript API. There is no API to search for contributors in JavaScript.

A Java based attribute customization plugin would have more API access and can provide better control and automation.

A real custom mail notification is possible using the following approaches

  1. Custom scheduled asynchronous tasks (see Due Date Notifier).
  2. Custom follow up actions that trigger at work item save.

The last three options require to write custom Java extension plugins and their deployment on the application server, which makes it less attractive in a cloud environment.

As an alternative, EWM has other mechanism that might be better suited. For example there are RSS feeds for various change events, work item queries and dashboards. These do not send e-mail, but they might scale better than e-mail notifications.

Keep in mind that the e-mail notification options are controlled at user level by the user. A real Java Eclipse plugin that sends custom e-mail notifications could work without the need to be correctly configured at user level.

Popular: Calculation with work item attributes

This is often requested to calculate effort or something similar.

With respect to JavaScript based attribute customization, it is possible to perform calculations on attributes of the same work item. It is possible to perform numerical, boolean and other operations in calculated values. It is necessary to set the dependencies right, so that the chain of events executes until all the calculations have been performed. If multiple work item attributes need to be set, keep in mind, that the calculated value is a function that only works for one work item attribute. If multiple attributes need to be set, a calculation for each attribute is necessary. As already mentioned, It is not possible to calculate across values of multiple work items.

If it is necessary to calculate across multiple work items, JavaScript attribute customization does not work. It is necessary to use the Java plugin extension mechanisms such as follow up actions to implement this. An example is the RTC Update Parent Duration Estimation and Effort Participant.

It is also possible to use external tools based on API to collect data and then update work items e.g. in scheduled jobs.

Popular: Calculate how Often a work item was in a specific state

I have seen this several times. There is no easy way to detect work item changes. It might be possible to use an approach as shown below to detect this, but I am not sure if this idea below would really work.

With some tricks and several hidden custom attributes for state tracking ST1, ST2 and counting STC, it might be possible to cobble something together. E.g. initialize the hidden state tracking attributes st1 and st2 with a default based on the state ‘New’.

The calculated value of ST1 is triggered and calculated based on the current value of the work item state attribute. If the current state is not the value of ST1 return the new current value.

Calculate the counter based on a triggered change of ST1. If ST1 contains the desired state, return the current value of STC increased by one.

Calculate the value of ST2 when ST1 was changed. Return the value of ST1 as result to the calculated value.

This requires some intelligent usage of the dependent attributes. It might be necessary to use more artificial attributes to get this working. This attempt has however many downsides, especially the custom attributes are all showing up in the history and also create more changes in the work item.

It would be easier to develop a follow up action that actually works on the state change, if this is necessary. It would be important to see if reporting can be used instead.

Popular: Make work item attributes read-only or required

EWM has preconditions/Advisors to make work item attributes read only or required based on conditions. The question is, how far can you go with this? The JavaScript API is very limited in what data is available. It is obvious that conditions can work on this data. So you can make attributes read only or required on work item attribute information.

But since you have no access to any complex information such as team membership or roles or the current contributor, JavaScript can not be used to create more complex conditions. Any condition that needs to have access to more complex information, would require to be written as Java plugin. An example requested by a customer with additional background information can be found here and in this post. It is important to know that these condition based attribute customizations are the only way to customize rules to make attributes read-only or required. There is no other extension point available that you could use to hook into. This is one of the rare cases where custom preconditions/Advisors can not be used.

In my experience with requests like here and in this post, to make attributes required or read only, are often motivated by the desire to have more fine grained control. Consider to use permissions to prevent roles from changing attributes.

Requests like above are also often motivated by the desire to make EWM work like some legacy tool. My experience so far has always been, that trying to make a tool work against its design is usually fatal. It usually makes the process hard to understand and follow. It also often degrades usability and performance.

Since every user in EWM has at least the default role (Everyone), can have many roles, and the roles have an order, it is also debatable what ‘by role’ really means.

Popular: Dynamically Hiding Work Item Attributes

I have also seen several requests that want to make attributes hidden by some complex, usually role based rule. This is not possible today, regardless the approach.

There are very limited capabilities to control visibility built into EWM. Attributes can be hidden if they are empty, if the work item is in a workflow state, if some endpoint is empty, if there is no project link and if the work item is just created. An editor presentation tab can be hidden, if all contained attribute presentations are empty.

There is no extension point, and there is no attribute customization available, I am aware of, that would allow customize the behavior in any other way.

There are various enhancement requests for EWM in this context. I think it would be nice if there was a custom attribute customization option that could be used with a precondition e.g. ‘Hide for condition’ that could be used similar to ‘Read only for condition’. This would allow to hide attributes that are unimportant. The question is, if it is helpful when attributes show up and vanish for the same attribute type. This is at least debatable. As a user I would likely find this confusing. As always, keep in mind this could open the door for performance issues and abuse. The remark about roles and their usage in EWM also applies.

Please note, just because an attribute is not visible in the work item editor, presentation does not make it inaccessible by work item queries or reports. EWM was also designed for collaboration and not hiding.

Popular: Dynamic workflow

I don’t know where this comes from. I doubt this is a thing in other tools for change tracking, but it cam up more than once.

EWM/RTC does not support dynamically changing the workflow of an instance of a work item. The workflow is tied to the work item type. It would be possible to use a follow up action/Participant to change the work item type under certain circumstances. Due to the lack of support for detecting a state change and the lack of access to other data in attribute customization, it would be not feasible to use attribute customization. A participant is a far better option.

Popular: Dynamic creation of attributes

This came up several times.EWM does not officially support this. It is possible to create an attribute using the API, but this attribute does not surface in the admin UI. It is necessary to add statements to the process XML to make such an attribute configurable in the process and the editor presentations.

This is used by some integrations just once and not meant to be used constantly during operations. Also mind the limit of custom attributes and consider usability of the process.

Restrict Read Access To Data

Some customers want to have more control about who can access which data. EWM has the capability to restrict work item and SCM item read access permission based on access context. If an item can not be read, it can also not be written.

For work items, the restricted access context can be an access group, project area, or team area. It is possible to automate setting the work items restricted access based on categories. This sets the access context to the process area associated with the work item category. This is the only built in automation for category based work item access control. There is no other built in automation for work items e.g. enabling the automatic selection of an access group for the work item.

For SCM data there are more options to set the access context/visibility. Components can have access control set to a project area or an access groups. SCM content in a component, folders, files, can be configured more fine grained. It is possible to limit access to a contributor, a project or team area or an access group. There is no built in automation to set the access context for SCM items.

It is possible to create custom automation to set the access context for work items and SCM items. One implementation approach is using follow up actions. For work item restricted access this can be configured for the work item save operation. For SCM objects follow up actions for the check in and deliver operation could be developed to compute the desired access context. I have explained some of the options in the blog series Setting Access Control Permissions for SCM Versionables.

In contrast to my knowledge back then, I know today, that it is possible to elevate the operation context to an administrator, why this is important is explained in the second last paragraph.

The question would be, can JavaScript attribute customization be used for custom automation for the restricted access? Attribute customization is only available for work items. There is no such API for SCM content. It might be possible to set the restricted access attribute in JavaScript attribute customization by returning the UUID of the access context, but as already mentioned, because of the very limited amount of data and API access available, JavaScript would most likely not be feasible for anything beyond that.

Java Based attribute customization could be more feasible, but it is most likely not possible to elevate the operation to super user access due to the limitation of the available API to the common API. I would consider this approach to be unfeasible for any relevant complexity.

If it is necessary to develop a Java Eclipse plugin for attribute customization anyway, it looks to be more suitable to create a follow up action to automate setting the access context. This allows to use the full capabilities of the EWM SDK common and server API and it is possible to elevate the operation to run in an administrator/super user context.

The reason why it would it beneficial to run the operation in an administrator context is, that the operational behavior usually runs with the permissions and limitations of the user that performs the save operation. This also means the operation fails, if the permissions are preventing it. This limits what can be done. For example it is impossible to change the access context of an item in a way that makes it inaccessible to the user performing the operation. The operation would fail. Elevating the operation to run in an administrator context, would allow such an operation to be successfully executed, regardless of the repository access and permissions of the regular user triggering the operation.

I would suggest to not try to use attribute customization for these use cases.

Summary

As a summary, attribute customization, especially using JavaScript can be used for some interesting purposes. There are however a lot of limitations in the JavaScript API that prevent several use cases. The biggest benefit is, that it can be used without having to deploy anything on a server. This allows it to be used in cloud deployments, where the users have no access to the server file system.

For more complex problems, the Eclipse plugin Extensibility is a better option, regardless if it is the Java based attribute customization, or Eclipse plugin based EWM extensions, just because the APIs provided in the EWM SDK are so much richer.

This was quite some effort. As always, I hope that this is useful to someone out there.

Advertisement

Debugging EWM Work Item Attribute Customization JavaScript

I don’t know why I have delayed this post for so long. Anyway, I am working on some workshop material and part of the workshop is RTC/EWM Work Item Attribute Customization using some simple JavaScript scripts. I often see questions about this type of attribute customization on the Jazz.net forum. Since it is pretty much impossible to remote debug scripts for others for various reasons, I want to provide some short introduction how anyone can use the built in Developer Tools of a Firefox or Chrome browser to debug the scripts.

Open the Project Area

Pick a browser. The latest Chrome or Firefox will do. Open the project area in the browser.

Debug mode

The JavaScript code is compressed when it is executed in the browser. To be able to see the scripts code as it is executed, it is necessary to enable the debug mode. This is done by injecting

?debug=true

into the URL used to access the CCM application. For example access the Work Items menu of the project area that has the attribute customization. Into the resulting URI below

https://elm.example.com:9443/ccm/web/projects/JKE%20Banking%20(Change%20Management)#action=com.ibm.team.workitem.viewWelcome

Inject ?debug=true in front of the hash # like so:

https://elm.example.com:9443/ccm/web/projects/JKE%20Banking%20(Change%20Management)?debug=true#action=com.ibm.team.workitem.viewWelcome

Make sure the page loads and the injected code is still showing in the URL.

Note: do not confuse ?debug=true with ?internal=true, which is for a completely different purpose (see the EWM Extensions Workshop the deployment lab).

Find the Scripts in Chrome

Once you have done that, open a context where the scripts would be available. For example create a work item of the specific type of interest that has the attributes.

For Chrome use the three vertical dot menu close to the profile, navigate to the More tools menu and then select the Developer tools menu.

The shortcut is CTRL+Shift+I.

The Developer tools can dock in different locations. You can control where you want it to be docked using the Developer Tools Dock side settings.

I have found the scripts in different locations over the years, at least I believe so. I really used to struggle to find them in Firefox. In Chrome it was really easy to spot, at least in my environment which made it my go-to debugging browser. This location has recently changed for some reason unknown to me. But this time I was able to locate the scripts in the folder top/<publicURIroot>/ccm/web/projects.

To locate the scripts select Sources and select Page if necessary. Then expand the folders until the scripts become available in the folder top/<publicURIroot>/ccm/web/projects. Where <publicURIroot> is the public URI root of you ELM server, that is encoded in the first section of the URL you have open just now.

Click on the script you want to debug. The script opens in the editor and allows you to set breakpoints as shown in the image below.

This now allows you to step through the script and debug as well as inspect the code and the current values in the script instance. The next image shows the debugger pausing at a breakpoint of the script and the how to inspect the value of the parameter attributeId.

You can step over the script statement by statement and the debugger displays the values and call results in the debugger. This makes it as easy as it can get to understand what the script is doing and why (or why not).

Find the Scripts in Firefox

Open Firefox and open the project area you are interested in. Modify the URL like above with the ?debug=true statement. Navigate to the Work Item tab of the project area.

Open the application menu (three horizontal stripes on top of each other). Select More Tools and then select Web Developer Tools

The shortcut is also CTRL+Shift+I. You can control where the Web Developer Tools dock using the customization menu.

Dock the Web Developer Tools as you please. Then open the Debugger tab.

Unfold the nodes MainThread/<publicURIroot>/ccm/web/projects. Where <publicURIroot> is the same public URI root that we know and love.

Click at the script you are interested in to open it in the debugger. Place breakpoints as desired. This works exactly like it does in Chrome.

Create a new work item or open one with scripts you want to debug and use the Web Developer tools similar to the Chrome developer tools.

Possible Issues

You might run into issued if you try this. E.g. the projects folder does not contain any scripts. This can be the result of various reasons. Here what I have run into.

  1. Did you enable Attchment Scripts in the Advanced Properties of the CCM server?
  2. Check by using a default filled in example script. Maybe your script is too corrupted/wrong to be loaded. You might want to consider to check the log file if there are load issues.
  3. If you just changed the scripts and still see the old version, make sure to force a reload e.g. Using STRG+F5 or otherwise reloading the page to pick up the new process.

Summary

I am at a loss, why I have not published this so far. Better late than never, this is going to be my URL for questions in the future.

I hope this helps users out there to smooth their development for attribute customization scripts.

EWM REST API to Access Existing Work Item Queries

After looking into the EWM OSLC CM API, including the OSLC Query API, a natural question that comes up is: Is it possible to take advantage of the existing EWM work item queries? Is it possible to use that query mechanism and run existing work item queries to get the result set back?

I had to look into that for a customer and an internal project and where experiencing some difficulties. So I think it is valuable to share my experiences and examples that finally worked for me.

Context of the blog post is the series

This is the series of planned posts I intent to publish over time. Most of the examples will be EWM based, but quite a lot of the content applies to more ELM applications.

External Links

I used at least the following links for exploring this mechanism.

EWM Stored Query API Introduction

EWM provides its own REST API that supports finding and running predefined, shared or personal work item queries that are stored in the repository. The API allows to query for all the stored work item queries available. The API allows to run such a query and to access the result information to present the work items and their data. This API is EWM only as far as I can tell. It is not documented to be available for any of the other products. It is also important to note, this API is not an OSLC API, even if the query URL contains oslc as part of the path.

The examples for how to use built in query are nice to have, but I found the documentation was lacking essential information. Maybe it is jut me and it comes naturally to all the native cloud wizz kids. I struggled to get the examples to work. One pattern I have now seen several times in documentation and examples is a lack of information about required and supported headers and how to URL encode query parameters. I will try to explain what worked for me. The documentation has recently been updated and now contains more details compared to when I first explored it.

For this specific part of the API, my observation was that getting the query parameters wrong, often resulted in the query returning results, but not the expected results. Usually the result set way way bigger than expected in those cases.

One important behavior that is very sensitive to the Accept header is how the information is returned and presented in the responses. There are two completely different pattern.

  • Only the URLs to the results are available in the response. To get any details about the result referenced by the URL, it is necessary to query that URL. This corresponds to the situation for OSLC queries without or an empty oslc.select statement.
  • Additional information is provided inline in the response. It is not necessary to query the result URI to be able to access the most relevant information. This corresponds to the situation for OSLC queries with an oslc.select=* statement.

When I looked at the API first, I missed information about the valid headers. As far as I can tell, the valid headers are now updated in the wiki page. The supported and working headers are mentioned below as well.

Headers and performance

Which header to use seems to be a quite unimportant difference, but it can have a huge impact on the communication performance. Lets create a Gedankenexperiment. Lets assume we perform a request to the server. We want to search in the response data to get the URI to a query with specific property values like a project area, query name and creator. Lets assume the following characteristics for the request:

  • Result set size is 130 items
  • Paging size is 50 items

To process the response with the properties inline it is necessary to run 3 requests at most. If the desired information is on page 1, only one request would be necessary. Worst case all related data needs to be sent and received.

To process the data without the properties inline it is necessary to load a page. Then it is necessary to run another request for each of the URLs in the result set page and process the received data. In worst case this requires 3 + 130 = 133 requests. Worst case all related data needs to be sent and received as well.

Why is that a problem? Worst case, if the desired data is in the last link processed, both methods have required to transfer the same amount of data. In many cases one could argue the 1st approach would even require more data to be transferred on average than in the 2nd approach.

However, in my experience the information transfer between two network nodes is significantly slower than the time that would be required in a server to get the desired information. Each message that needs to be sent causes a considerable amount of overhead and delay. It is usually cheaper to send fewer messages than many small ones that transfer a similar amount of data, just because of the overhead required for each message. In addition all the subsequent requests for the details cause activity on the server which is likely to cause a lot more overhead compared to collecting all the information for the 3 initial request(s).

As an real world example, when I started to look into this, I did not know which headers where available. I used the headers I had often used e.g. ‘application/rdf+xml’. In addition, the query that I created had an issue. Instead of only returning the queries for a specific user in a specific project area, it returned all queries for the whole server. The responses only contained the query URLs. To get the name and creator of the query each URL had to be requested. It was also not obvious why the result set was so large and I had no code to recognize that results were related to project areas other than the expected. The server I ran against is on another continent and I was in my home office. I stopped the experiment after an hour or two.

At the end I was able to create solution that performs well and reduces overhead to a limit. But it was no means a simple process. I reached out to someone for hints. As a customer I would not have had the opportunity. This is why I try to finally share this information.

Stored Query API URL

I was not able to find a way to discover the URI to get the work item queries. The Wiki page mentions to find the simpleQuery tag in the project areas service provider, but I seem to be unable to find it, so I need to construct it based on the wiki page. From the Resource Oriented API page I can deduce two possible forms for the URL. Assuming PublicURI being the public URI of the CCM server e.g. ‘https://elm.example.com:9443/ccm&#8217; the URL for the Stored Query Collection is either

PublicURI + '/oslc/queries'

or

PublicURI + '/oslc/queries.xml'

Both seem to work. Choose one you like and that works for you.

Query Stored Queries

The request below gets all the stored queries for a repository.

Get all stored queries for the repository

Note that the only Accept headers documented to be valid at the moment are:

Accept text/xml
Accept text/json

It is necessary to be authenticated to the ccm server to be able to perform the operation.

The image below shows a part of the response to the query above.

The response to the query for all stored queries

The first section shows the query for the next page in the attribute oslc_cm:next and the total result count in oslc_cm:toatlCount.

The rest of the information is one element rtc_cm:Query for each query descriptor. The element contains the query URI as rdf:resource, the identifier for the query, the Query name/title and description and the project area the query belongs to. The most important information is the rtc_cm:results element with the URL to execute the query. As an example the URL below:

https://elm.example.com:9443/ccm/oslc/queries/_87dqoFpmEeukW7cqqDjAuA/rtc_cm:results

Additional information is the date of the last modification and the user that created the query.

Note that the result is either XML or JSON. The tools for RDF that were used in some of the previous posts using the Python rdflib do not work in this context. Instead the Python code shown in the blog uses xml.dom.minidom to access the information in case of XML. The JSON code examples shown in previous posts using the json library can be used as a reference. This post will mostly use the Accept header text/xml. The code depends on the following libraries.

The main Libraries used to process the stored queries API.

Some example code for processing the data in Python will be shown below.

Narrow Down Stored Queries

It is usually not desirable to query for all stored queries for all project areas. A real production server can have hundreds of project areas, team areas and users that can have predefined or shared or personal queries defined. The result could be thousands to ten thousands of queries. To collect and transfer all that data is a challenge and will drain server resources as well.

It is possible to filter for all the properties that make up a query, identifier, name, creator, owning project area. The code below computes the query parameters from the parameters passed and creates a query section for the query URL.

Create the query parameters to narrow down the query results.

As an example the query that is created could look like below. The condition can narrow down to the project area, the user and the title, dependent on how much information is provided. If enough information is given, this would usually only return one hit. Note that this is a string terminated by the singe quote and the double quote characters inside are important. The query narrows down to one project area, the query name ‘All’ and the creating user.

'rtc_cm:projectArea="_8e5qfFpmEeukW7cqqDjAuA" and dc:title="All" and dc:creator="https://elm.example.com:9443/jts/users/ralph"'

How to discover the project area UUID has been explained in previous posts e.g. about discovery. The code that was used here is the same used in the other posts.

The code blow shows how the query URL is composed.

Python code to compose the query parameters and to create the query URL

The parameters provided are the same as above, the project area, the query name and the query creator. The creator can be provided in different ways. It is possible to provide a user URI like

'https://elm.example.com:9443/jts/users/ralph'

It is also possible to provide the variable string below.

'{currentUser}'

The query mechanism replaces this variable with the URI of the user that runs the request. For example, the query below would return the query named All created by the current user.

'rtc_cm:projectArea="_8e5qfFpmEeukW7cqqDjAuA" and dc:title="All" and dc:creator="{currentUser}"'

The last three lines in the image above compose the query URL. First the base query URL is composed. This is the part of the query that returns every stored query. Then the query parameter section is composed by adding ‘?oslc_cm.query=’ and then adding the URL encoded query parameters. The resulting query URL looks like below.

'https://elm.example.com:9443/ccm/oslc/queries.xml?oslc_cm.query=rtc_cm%3AprojectArea%3D%22_8e5qfFpmEeukW7cqqDjAuA%22+and+dc%3Atitle%3D%22All%22+and+dc%3Acreator%3D%22%7BcurrentUser%7D%22'

Execute the Query for Stored Queries

The code below executes the query to search for stored queries.

The first line runs the query that was just created using the function findQueryResources().

Execute the query to find stored queries

This function is shown in the image below. It uses the Accept header ‘text/xml’. While there is a valid query URL for a page, it executes that query.

The code that executes the query for the stored queries and collects the results

It calls qeryQueryInfoXML() to process the page result data. This code is shown below. The function builds up two arrays, one containing the stored query URL and one the associated query name. It also gets the query URL for the next page, if there is any, and returns that.

Get the information about the queries from the response

All query pages are iterated and the results added to the respective arrays.

Evaluate the result and

At the end of executing findQueryResources, there is a list of query names and an associated list of query URLs. Dependent on the query parameters given, it could return many or no result. The code here as shown below focuses on using the query name, but it would be easy enough to extract more information per stored query to use different criteria to look into the result set. The code tries to find the first stored query returned with the given name.

Find the query to run and execute it

If a stored query has been found, that query is finally executed in the function execute_query_xml(). The code for this function is shown below. Again, this uses the Accept header ‘test/xml’ and the results are processed as XML.

Process each result page of the stored query

The code iterates the query pages and is similar to the code already shown above. The code passes each query result page to the function processQueryResultInlineXML(). The code is shown below. It also looks very similar to the code we have already seen. The difference is that the code below now works on work item resource information and not on query information.

Processing one page of the stored query with work item data inline.

The code analyzes the oslc_cm:Collection and gets the total result count and the next page (if there is one). Then it analyzes the inline XML data for all the work items. For each work item it gets some of the available information and prints that. It returns the work item URIs, the totals and the next page etc.

Executing Stored Query Using JSON

The code here shows a JSON based implementation of the functions to execute the stored query after finding it. The function execute_query_json() does the same the function execute_query_xml() does and looks very similar. It is only using JSON as format to parse the data. The Accept header used below is ‘text/json’.

Process all query pages in JSON

The function getQueryResultDetailsInlineJson() is called to process each page. Like its XML twin it gets the total count and the next page (if available). Then it gets details about the embedded inline work item data and logs it. Like the XML based code, the function builds an array with the work item URIs for the page. It also returns additional information such as the link to the next page.

Process one query page with JSON work item data inline

More information

Please check the section Getting Resource Representations to understand additional ways to get work item data representations. Also check the section Getting Partial Representations to understand options to limit the data that needs to be transferred.

CSV Export

The query descriptor URL provided in the search by the tag rtc_cm:Query, can also be used in a browser to get the query displayed. As an example see the information below.

<rtc_cm:Query rdf:resource="https://elm.example.com:9443/ccm/resource/itemOid/com.ibm.team.workitem.query.QueryDescriptor/_pdT5AATBEeyiDIlAjMF45Q">

Opening the URL in the rdf:resource results in the following display in the browser.

The query descriptor URL is opened in the browser and executes the query.

It is also possible to use that URL to download the query result as CSV file. By adding the query parameter media type e.g.

?_mediaType=text/csv

to the query descriptor URL, the query can be changed to export the data as CSV file. As an example the URL below exports the query result to CSV.

https://elm.example.com:9443/ccm/resource/itemOid/com.ibm.team.workitem.query.QueryDescriptor/_pdT5AATBEeyiDIlAjMF45Q?_mediaType=text/csv

By pasting the URL in the browser, the CSV download is triggered. It is also possible to use the URL above in tools like cURL.

Summary

This blog post has described the steps that are necessary to find existing work item queries and how to run such queries to get the data. I hope that the examples here in my blog help users out there with their work.

Searching and Exploring the RTC SDK

I have answered many customer questions about the RTC Java APIs in the past. Many of the posts in this blog are the result of such questions. I have no privileged access to the RTC source code. As far as I can tell, there is no secret library of RTC API documentation that the IBM development team is hiding from everybody else. Even if there was such documentation, I do not have access to it. So, how do I come up with those answers?

In short, by using the available capabilities to search for answers.

Prerequisites

The content below requires to know Learning To Fly: Getting Started with the RTC Java API’s and the Extensions Workshop.

Know where and how to search

The whole answer is, that it is not necessary to know everything. It is just necessary to know how and where to search for the answers. These days, there are basically two answers to where and how to search.

  1. The Internet using search engines
  2. The RTC SDK using the Eclipse IDE’s capabilities.

I have already explained how and where to search the internet in the post Learning To Fly: Getting Started with the RTC Java API’s in the section Where can I find Examples and Example Code?.

Search the internet

Search the internet and limit the search to sites that are likely to have relevant information. Examples are Jazz.net and stackoverflow.com. Jazz.net has the Forum and the development Wiki. Stackoverflow has a lot of questions and answers from many contributions.

I did contribute at stackoverflow as well, but gave up when they would not accept a link to external content as answer. The purpose of this blog is to be able to provide answers and share examples. For that very reason rsjazz.wordpress.com is also a good place to search. The search capability in the top right of this blog is also a valid place to search. The page Interesting Links is a collection of links to other sources I have come across over the years.

Very recently the Jazz community and Github have become a source as well.

Many questions in the context of RTC Extensions might be just related to Eclipse Plugin development. https://www.eclipse.org/ has a lot of examples and documentation around such questions.

Search the RTC SDK

If there are no ready examples and answers in the internet, there is a host of example code available in the RTC SDK. The RTC SDK contains at least the following:

  1. Unit test code for RTC the Java API. These contain especially examples for the administration API, but also for other parts of the client API provided by the Plain Java Client Libraries.
  2. The Java Code for the client API provided by the Plan Java Client Libraries, including the documentation that is used to create the JavaDoc for the Plain Java Client Libraries.
  3. RTC Rich Client Platform code used in the RTC Eclipse Client, including comments. This code uses the public client API but also uses internal API.
  4. RTC Client plug in code, including Client operation behavior such as advisors (pre-conditions), participants (follow up actions), aspect editors providing the Eclipse Admin UI and related information.
  5. RTC Server API and RTC Server code, including documentation for interfaces and methods.
  6. RTC Server plug in code, including server operation behavior such as advisors (pre-conditions), participants (follow up actions), aspect editors providing the Eclipse Admin UI and related information.
  7. Client and server extension points and related code.
  8. Code for asynchronous tasks.

All the code mentioned above is included in the RTC Server SDK that is shipped with RTC in the all download section. The RTC Client SDK that is shipped in addition since 6.0.3 only contains the client code which is kept compatible to earlier versions of the Eclipse client.

You can search the RTC SDK by using the capabilities provided by Eclipse. Especially the Eclipse Plugin Development (PDE) and Rich Client Platform (RCP) development tools provide a comprehensive set of features to search the RTC SDK. There are several approaches that can be used to search in Eclipse and the Eclipse PDE:

  1. Java Search for types, interfaces, methods and other Java Related properties. This includes the capability to use search pattern e.g. use an asterisk. It is possible to specify the relationships to search for e.g. search for classes that implement an interface.
  2. Search for Plugins, Extension points, references to extension points. This includes the capability to see the plugin.xml for other extensions, revealing all the implementation classes and relevant information that can be used again in 1 to narrow down the search to the relevant code.
  3. Eclipse RCP/SWT Inspectors such as Yari or tools like the built in Plugin Spy allow to use the declarative and reflective capabilities of Eclipse and the RCP to analyze the UI and other information provided in the plugins and features. This allows to find which classes are called in menus and to look at which data is used by views and how it is used.
  4. Other search capabilities such as File search and Text search can help in certain conditions.

These capabilities allow to pretty much find anything that is available in the RTC SDK. This does not make it trivial to understand what is found, especially the RCP UI code of RTC is sometimes very hard to understand, but it provides at least entry points, where to start.

Once something is found, there are various means available to

  1. Open or navigate to the element that is found
  2. Open the package containing the element, if applicable
  3. Open other items that are related to the found element

This is really all one needs to know. The rest is just using these capabilities to your advantage. Some examples how to use this are given below.

Finding Built-in Extensions in the SDK

The SDK contains all the client and server extensions and their source code. When developing a custom advisor or follow up action e.g. following the Extensions Workshop it is a good idea to get inspiration from the existing code. How does that code work? How does it use the API?

Any such scenario starts with finding examples that use one of the relevant extension points. The extension points are shown in the Extensions Workshop and other examples. Most likely there is an example using an extension point in the plugin.xml already, or a new plugin.xml has been just created.

In the open open editor of the plugin.xml on the Extensions tab, right click the extension point that is of interest. To understand where this is used select Find References and click to start the search.

Search for references to an existing extension point.
Search for references to an existing extension point.

The search window will open below and show the references found after a while. This can take some time, dependent on the performance of the client computer. The search result will show multiple hits. The image below shows a part of the hits in the current workspace. It especially shows the references from RTC SDK code to the extension point used by server operation advisors. All the references shown here have the same namespace prefix com.ibm.team. This hints that the origin of the code is in the RTC SDK.

Operation Advisors shipped with RTC.
Operation Advisors shipped with RTC.

Use the up and down arrows to browse the referencing plugin XML’s, or click one of the rows in the search that is of interest. Make yourself familiar with the name spaces that are used. This makes it easier to focus on the important examples you are looking for.

The namespaces refer to the following components of RTC

  • com.ibm.team.apt – Agile Planning
  • com.ibm.team.build – Build
  • com.ibm.team.filesystem – Source Control Filesystem
  • com.ibm.team.scm – Source Control
  • com.ibm.team.workitem – Work Items

When looking for an example for SCM related operations such as deliver, look at the examples in the namespaces filesystem and scm. When looking at work item related extensions look into the examples with the workitem namespace.

Click the search result to open the related plugin.xml. As an example the search result
com.ibm.team.workitem.service. Eclipse allows to find strings in open files, so it is possible to search for strings using CTRL+F. E.g. search for Required to find because you are interested in advisors for required attributes.

Required Attributes by Tape and State Advisor
Required Attributes by Tape and State Advisor

In the plugin.XML you can see the implementing class. You can use the full qualified class name e.g. com.ibm.team.workitem.common.internal.RequiredAttributesByTypeAndStateAdvisor to open the class using the Navigate>Open Type menu of Eclipse.

Open the implementation class of the advisor
Open the implementation class of the advisor

This repository workspace has the SDK set up as well as the RTC Plain Java Client Libraries. Select the class in the SDK. Browse the classes code and documentation. Open the Class in the package explorer.

Explore the package containing the class
Explore the package containing the class

In the package explorer view in Eclipse browse the package e.g. C:\RTC605Dev\installs\rtc-server-sdk\plugins\com.ibm.team.workitem.common_3.2.900.v20170928_1641.jar.

Scroll down to the lower area. There are some packages that do not contain Java classes. For example there is a package or rather folder, that contain files such as the schemas used by the aspect editors in the admin UI. You can open the files to look at the details.

Package explorer shows related content.
Package explorer shows related content.

Please note: the one I was really looking for is a bit elusive, and I have not been able to locate it until now. I will update the post as soon as I have a solution.

This is only one way of doing it. Dependent on what is available and what one knows there are many other ways to look into the source code. For example search for all extension points with a package name of com.ibm.team* to find all the extension points. Search for specific Plugins, in case the name is available.

Search for References to an extension point.

Try to explore different angles to find what you are looking for.

Search the Java Code

As already mentioned, it is possible to search for Java classes. This can also use search patter using an askerisk. Search for different aspects such as methods and interfaces and limit the search result set.

Once a class is open, it is easy to navigate within the class hierarchy, open the jar file in the package explorer etc..

Explore the UI to add Menus

Here a key point is knowing and using Eclipse RCP/SWT Inspectors such as Yari. How it is done is This has been explained in Adding Context Menus for Jazz Objects to the RTC Eclipse Client and Hiding UI Contributions in the RTC Eclipse Client.

Summary

As seen above, there are many ways to find stuff in the RTC SDK that can be used. As always I hope this helps users out there with their endeavors. I will try to add more examples.

Work Item Command Line 4.3

I worked with a colleague on the Work Item Command Line who was adding a feature for usage with a customer. That colleague had also done some changes related to availability and performance running queries. These changes where merged back into the code. WCL was missing some capabilities that I always wanted to have. I had a quick look to understand how much effort hat would be and took the opportunity to implement these capabilities on the way. While doing so, some small enhancements where added and some defects where found and fixed. All that needed to be published as a new version of WCL.

Print a work item

New in Version 4.3

The latest release 4.3 is available here and has the following changes:

  • Readme.md updated
  • Readme documents for deployment in source code updated
  • Refactored the capability to create a string representation for a work item attribute, link or pseudo attribute (e.g. attachments) into a helper class WorkItemExportHelper
  • Migrated -exportworkitems to use the new helper class WorkItemExportHelper
  • Several fixes in -exportworkitems to
    • Enable error handling and make suppressing errors work
    • Introduce a new flag that exports all supported columns and links
    • If exporting all columns, order the attributes and links (except ID, Type and Summary that are by default at the beginning)
    • Checked functionality with newest OpenCSV
    • Refactoring class to make maintenance easier
  • Added a new command -printworkitem that prints the attributes, attachments and supported links of a work item
    • Using the WorkItemExportHelper and its capabilities
    • Also provides switch to print all attributes and links ordered as above
    • Exports the attachments to a provided folder
  • Added a new command -printtypeattributes that allows to get the attribute Id’s and Display Names for a given work item type in a project area
  • Added a command -validateoslclinks that validates OSLC links and their back links in other applications fixing missing back links
  • External library dependencies are added without a version number to allow newer versions

Disclaimer

Please keep in mind that the information in this blog is “as is”, unsupported, and may be outdated or inaccurate.

The Work Item Command Line (WCL) is not an officially supported application.

For information on released products, consult the product documentation, support tech notes, and the Jazz.net library.

Open Items

Some of the API used in the Work Item Command Line becomes deprecated.

  • Some changes especially for -importworkitems needed for versions of RTC e.g. 6.0.5 and later
    • These will likely not work with earlier versions of RTC
    • if you need that either try an earlier version of WCL such as 4.0 or 4.1 or open an issue and I could look into a backport
  • The Linking API in RTC has evolved and deprecates some API used in WCL
  • OpenCSV has evolved and newer versions deprecates some API used in WCL
    • OpenCSV requires a new library commons-lang3-3.1.jar
  • Migration to Maven might be something to consider

This is all not problematic yet, but needs to be addressed sooner or later.

License

WCL is released under the MIT License. See the License.txt and the license headers in the individual files.

Compatibility

This code has been used with RTC 4.x, 5.x  and 6.x with no or minimal changes and it is pretty safe to assume, that the code will work with newer versions of RTC to come.

An exception is the -importworkitems command that requires a library to use a mapping file. The capability to use the mapping file was moved into the plain Java Client libraries in RTC 6.0.5 as an internal class. To support work item import with the mapping moved to the Plain Java Client Libraries as internal API, the imports in ImportWorkItemsCommand where changed and the file com.ibm.team.workitem.rcp.core*.jar was removed from the jar file dependencies in the build path.

See Restore Compatibility to 6.0.4 and before for how to switch this back.

The code requires the RTC Plain Java Client Libraries.

The Export and import commands require additional external libraries that need to be downloaded and installed separately. See the document below.

Restore Compatibility to 6.0.4 and older

To support 6.0.5 and beyond the imports in ImportWorkItemsCommand where changed from:

com.ibm.team.workitem.rcp.core.internal.bugzilla.mappers.BugzillaMapping*

to: com.ibm.team.workitem.common.internal.importer.bugzilla.mappers.BugzillaMapping*

and the file com.ibm.team.workitem.rcp.core*.jar was removed from the jar file dependencies in the build path.

Follow the description in the sections beginning with Downloads how to get and rebuild the code. The steps in this section below are required to port WCL back to RTC 4.0.4 and earlier:

The original imports have been put back in past 4.0.3, but commented out. To enable the mapping for earlier versions of RTC, replace the imports *BugzillaMapping* with the versions commented out. Then add the com.ibm.team.workitem.rcp.core_*.jar file back in to the build path and add the required JAR file as below

From Work Item Command Line 4.1

The import commands of WCL can only provide the capability to use a mapping file by using a JAR file that only ships with the RTC Eclipse client and the SDK. The classes used for the mapping file capability are located in the library com.ibm.team.workitem.rcp.core.  Open the Install location of the RTC Eclipse client and search for com.ibm.team.workitem.rcp.core*. You should find a file names similar to this one: com.ibm.team.workitem.rcp.core_3.1.900.v20141010_0124.jar. The version numbers at the end could be different. Copy the JAR file into the into the libfolder of your version of WCL.

Download

The latest code can be downloaded from this GIT repository. The latest releases can be downloaded from here

There are different ways to download.

  • The easiest is to download the latest code as a zip file using the button Clone or download.
    • Use the option Download ZIP to download the code as ZIP file. Once the download finishes, browse the folder structure of the ZIP file. The top level contains the README.md file. An Eclipse project is contained in the folder com.ibm.js.team.workitem.commandline. This project can be used to develop the code and to generate an executable.
    • It is also possible to clone the repository and import the project using GIT. Clone the GIT project to your local file system. This requires GIT or some GIT GUI to be installed. How to use GIT is out of the scope of this blog. See the help how to set up GIT and how to use it with Eclipse.
  • It is also possible to download releases from the releases page. The zip code made available is in the same format as the one above can be used as described above. Note that the releases might be older, and there might be no release available for the latest code version.

See below how to import the enclosed project into Eclipse.

Some releases might contain a prepackaged executable for convenience e.g. named wcl-V4.3-20190117.zip. See the section Deploying the packaged application below for how to use this form of packaging. Please note that the prepackaged executable might not be available for all versions. The following sections explain how to create this executable. See the section Packaging the application for shipping for how to perform the packaging.

Prerequisites

Install a current Eclipse or RTC Eclipse client.

Install a current Java 1.8 JRE or Java 1.8 JDK set it as active in Eclipse as Installed JRE and Execution Environment.

Getting the code, install and deploy

Once you have installed such an environment do the following

Import the Code

  • Open the Java Perspective in Eclipse.
  • Import the project into Eclipse from the file system.
    • To import the ZIP file use File>Import, select Existing Projects into Workspace. Click Select archive file, browse to and select the ZIP file you downloaded. Import the enclosed Eclipse project com.ibm.js.team.workitem.commandline.
    • To import the GIT repository use the GIT perspective to add the local cloned repository and then import the
      project com.ibm.js.team.workitem.commandline.

Create a User Library for the Plain Java Client Libraries

  • Use Window>Preferences>Java>BuildPath>User Libraries and add a user library named PlainJavaApi .
  • Add the JAR files of the Plain Java Client Libraries for the version of RTC you are using.

Download and Install openCSV

Download and Install Apache commons-lang

Check the project and recompile

  • Select Project>Clean and clean the project that was just imported.
  • There should be no compiler errors or issues with the classpath

The project should now have built and compiled and show no errors.

Building the WCL or extract from a Release

Follow the description in the file ReadMe – HowToRelease.txt in the root folder of the project

  • To create a wcl.jar file
  • To crate a WCL/lib folder
  • To provide batch and license files

Follow the description in the file /lib/ReadMe.txt in the lib folder of the project

  • To provide the required libraries in the WCL/lib folder

Select the folder, for example C:\Temp\wcl\ and compress the file

  • Rename the archive file to wcl-Vx-YYYYMMDD.zip,
    where YYYY is the year, MM is the month and DD is the day
  • The file is now ready for publishing

Use the file wcl-Vx-YYYYMMDD.zip for shipping similar to the next section.

Deploying the packaged application

Assume a file was created following the steps above and the file is called
wcl-V4.3-20190117.zip. This binary contains WCL as JAR file and other information such as licenses and Readme files. It also contains script files to set the environment and run the Jar File.

The folder lacks libraries needed to execute that need to be provided.

To deploy from this file, create a new folder as deploy location for the automation prototype. In the description here we use C:\temp. It is possible to use any other folder. In this case replace the folder in the this description by the folder you chose.

Extract the binary wcl-V4.3-20190117.zip file into the location C:\temp. There should now be a folder C:\temp\WCL. The folder among others contains the files wcl.jar, wcl.bat, wcl.sh.

Get the libraries as described in Download and Install openCSV and Download and Install Apache commons-lang above, and rename them as described in /lib/ReadMe.txt.

Check the script file and make sure to provide a valid path to a folder containing the unzipped Plain Java Client Libraries and a recent Java 1.8 is available over a Java_Home.

Call WCL using the script files or call it as Jar File as described in said script files.

Just starting with extending RTC?

If you just get started with extending Rational Team Concert, or create API based automation, start with the post Learning To Fly: Getting Started with the RTC Java API’sand follow the linked resources.

You should be able to use the following code in this environment and get your own automation or extension working.

Summary

This is only the first blog. I will try to provide a little bit more about the commands and the parameters soon.

As always, I hope this helps users out there. 

Please ask questions on in https://jazz.net/forum/questions/ and tag them rational-team-concert and wcl.

You can create issues in https://github.com/jazz-community/work-item-command-line/issues as well. 

Keep in mind this is off hours work and not officially supported.

Build On State Change Work Item Save Participant

In the unlikely event you have missed it or just to complete the hit list if you search for examples on this blog, the Build On State Change Work Item Save participant/follow up action is a complete example as part of the Rational Team Concert Extensions Workshop.

The Build On State Change Work Item Save participant monitors work item state changes of configured work item types and state changes. If a qualifying change happens, it issues a build request for a configured build definition. The example comes with a complete package including the configuration UI.

Just Starting with API work?

If you just get started with extending Rational Team Concert, or create API based automation, start with the post Learning To Fly: Getting Started with the RTC Java API’s and follow the linked resources.

RTC Process Customization – What you can and cannot do

Rational Team Concert Process Customization – What you can and cannot do, that is the title of the webinar I presented two days ago.

If you are interested in my view on this, you can find the replay of the webinar here in the Rational Team Concert Enlightenment Series.

The slides are shared here.

Also see What API’s are Available for RTC and What Can You Extend? for more links.

How to filter RTC email notifications by project or by user

Spamming the in-boxes of your project with notification mails? This is a reblog of Marks internal blog.

The RTC Mail Filtering Problem

A common requirement for Rational Team Concert administrators is the need for limiting email notifications for a single RTC project or user.  There are many scenarios which might drive this need.  Here are some examples:

  • Thousands of work items are imported and you want to not cause notifications for the new work items.
  • A new field is added and data from the old field is copied over, affecting many work items.  We don’t want users to be notified.
  • A subset of users now use a different RTC server, and they no longer want notifications from the old project, but they need to remain active in the old project.

Unfortunately, the current RTC product as of version 6.0.1 does not support this requirement.  Notifications are controlled at the Jazz Team Server level.  The JTS may control multiple RTC, Rational Quality Manager, and Rational Doors Next Generation repositories.  Turning off notifications in the JTS turns off mail for all RTC, RQM, and RDNG projects from all repositories controlled by that JTS server.  Mail  generated for all of those RTC, RQM, and RDNG projects while notifications are off in the JTS are lost.

This leaves administrators with a tough choice:  lose all mail for everything connected to the JTS, or live with excessive and unwanted notifications for a single project.

A Solution:  Milters

We have found and implemented a solution for this requirement, milters.  Milters are plugins for Sendmail that add additional functionality to Sendmail.  The “milter-regex” milter plugin permits filtering mail using regular expressions.

These are the overall steps for utilizing the regex milter:

  • Set up a machine with Sendmail and install the milter-regex plugin
  • Configure Sendmail to allow mail from the JTS server machine in /etc/mail/access
  • Develop a set of regular expressions which cause mail from a particular project or user to be found and filtered out and update the /etc/mail/milter-regex.conf configuration file
  • Restart the milter-regex service
  • Edit the email settings of the JTS to point to your Sendmail machine as the mail SMTP Server

You can easily add and remove projects and users from the filtering list by editing the /etc/mail/milter-regex.conf file.  You can turn off filtering completely by restoring the JTS SMTP Server value back to its original setting.

Regular Expressions

You’ll have to examine your email templates to determine the project and user information for the various email formats.  Here’s an example of the configuration statements for our email templates to filter out all mail for project “Project XYZ”:

reject “Mail filtered for project: Project XYZ”

body ,Team Area:.* Project XYZ,i

body ,Project Area:.* Project XYZ,i

Both “body” statements are required.  The “reject” statement defines a message which is logged into the /var/log/messages log file when a piece of mail is filtered out.

This is an example of filtering for a single user:

reject “Mail filtered for user: Mark E. Ingebretson”

body ,The user ‘Mark E. Ingebretson’ made a .* request,i*

body ,Mark E. Ingebretson mentioned you in,i

body ,Mark E. Ingebretson.*changed on,i

If you are a user of our IBM Systems servers and need email filtering, you can submit an ITHELP request.

Other Approaches

There is another approach from Sam provided on the Jazz.net forum here: Manage User E-mail Preferences for Mass Updates. It has been sitting on my Interesting Links page for a while. Time to show it here.

A new solution s available for RTC 6.0 starting with iFix03 if you are using the Java API. Use the Skip Mail Save Parameter.

Summary

I found the topic very interesting and related questions also come up on Jazz.net, so I decided to re-blog and promote this when Mark showed this to me. I hope it helps our users all over the world. I hope that this solution can help other RTC administrators address this important requirement.

JavaScript Attribute Customization – Where is the log file?

A lot of users try Java Script based attribute customization and often run into issues. They ask on the Jazz.net forum to get the issue solved. Unfortunately the questions usually lack the information required to help. This post explains how to retrieve log information to be able to provide this information.

Where are the Script Log files?

Java Script attribute customization can use the console to log text messages into a log file.

console.log("My message");

The question is, where are the log files?

The script context

Java Script attribute customization scripts are, as far as I can tell, run in one of the following contexts:

  1. The Eclipse Client
  2. The Web Browser
  3. The RTC Server

Dependent on the context it is run, the log information can be found in a log file that is created and maintained by the

  1. The Eclipse Client
  2. The RTC Server

Please note that the logging information is not in the RTC Application log file CCM.log.

The Jazz.net Wiki entry about attribute customization provides hints about how to log data and how to log data and how to debug scripts in the section Debugging Scripts. Similar information is provided in the Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management.

Unfortunately both only talk about how to find the server log information for Tomcat. Since Websphere Application Server and WAS Liberty are also valid options, how can one find the log files in this case?

Find The Eclipse Workspace Log

As background, note that the Eclipse Client as well as the RTC Server are based on Eclipse technology. This common technology is used to log the data and determines the log file location and name.

Each running Eclipse has a workspace location and stores meta data and log information in this workspace. The workspace is basically a folder in the file system. The metadata is stored in a sub folder with the name .metadata. The log file is in this folder and named .log.

For the RTC Eclipse client and for scripts that run in this context, open the Eclipse workspace folder that is used and find the .log file in the .metadata folder.

For the RTC Server, the easiest way to find this workspace and the enclosed log file that I have been able to find is to search for the folder .metadata. For Tomcat and WAS Liberty standard installs go to the folder where the RTC Server was installed and then into the sub folder server. From here search for the folder .metadata.

For Websphere Application Server (WAS) go the profile folder for the profile that includes the RTC server deploy and search there.

Here an example search for a test install based on WAS Liberty:

LogFileLocations_2016-06-17_13-10-14

Note that every Jazz application has its own Eclipse workspace with metadata folder and workspace log file. The one interesting for RTC attribute customization is the workspace of the RTC server. The folder structure includes the context root of the Jazz application. Each application has a different context root which typically matches the application war files name prefix. The RTC application typically has the context root and application war files name prefix ccm. Open the workspace for this application and find the log file.

Looking Into the Log File

You can look into the log file. Please make sure to use a tool that does not block writing to the log file, while you are browsing its content. The log file is kept open by the server when it is running and blocking it from writing is not what is desired. Use more or an editor that reads the file and does not block it. For windows users: notepad does lock the file for writing. Use a different tool such as notepad++.

The Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management provides some examples for how logs look like and can be created. If you can’t find the log entry you are looking for, always check the server log as well. Maybe the script runs in a different context than you expect.

Here an example for log entries:

Log_Examples_2016-06-17_14-58-43

Load Errors

It might happen, that an expected log entry is not found in any of the log files. In this case make sure to check for script loading errors as well as thrown exceptions at the time the script was supposed to run.

Load errors can be caused by different reasons.

One reason can be that attachment scripts are not enabled. There are enough indicators in the Attribute customization editor in Eclipse that a user should have spotted this these days.

Another reason can be that the script is syntactically not correct and can not be interpreted as a valid JavaScript. One reason for a script not being recognizable as a valid script that I have seen recently is an incorrect encoding. If an external editor is used to edit the script and the script is then loaded from the file, make sure that the script has a correct UTF-8 encoding. If in doubt change the encoding to UTF-8 and reload the script.

Why would the encoding be important? The encoding controls the format of the content. it is hard to determine the encoding from a file and it is often not checked. But expecting a specific encoding but loading a file that was encoded in a different one can cause to find unexpected content. This can can cause the JavaScript not being recognized as JavaScript and the load fails.

Debugging vs. Logging

Using the debugging techniques explained in the Wiki entry in the section Debugging Scripts and in the Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management should be the preferred option and is usually more effective.

Looking at the logs is still a valid option, especially to be able to log execution times and to find script loading issues and for scripts that are run in the background in the server context, such as conditions.

I found using the Chrome Browser and the built in Developer Tools to be most effective. The scripts can easily be found in the sources tab under the node (no domain). Make sure to enable the debug mode as explained here: Debugging Scripts.

JavaScript_Debug_Chrome_2016-06-17_14-24-43

Summary

This post explains how to find the log files that contain log information written by JavaScript attribute customization scripts. I hope that this helps users out there and makes their work a little bit easier.

Check out the RTC Timebox Planning Widget with SAFe Support

Take a look at this great community contribution. If you ever wanted to be able to easily see the status of a plan and balance the load in the dashboard, this could be the ideal solution for you.

I already blogged about the great predecessor and other community content in the post Some Community Extensions for RTC.

In the newest post New RTC Timebox Planning Widget with SAFe Support, Markus Giacomuzzi from Siemens Switzerland explains the newest version of their RTC Timebox Planning Widget with SAFe Support. He also created videos on their You Tube channel, explaining how to use and set up the TimeBox Planning dashboard widget. The Time Box Planning widget is an excellent example for how community contributions can improve the useability of RTC and provide fresh ideas. Please read the post and view the videos. Don’t forget to share and give Markus a thumbs up.

The extension can be found here on IBM Bluemix DevOps Services, if I am not mistaken.

Markus is going to present this at the IBM InterConnect 2016 in Las Vegas in the session DOP-3145 and you might want to consider visiting the session to be able to see it and share your experiences and ideas.

Here is a screen shot of the program level planning.

TimeBox Planning with SAFe support program level

And here is a screen shot of the team level planning.

TimeBox Planning with SAFe support team level

Last but not least, I thought about delaying this reblog for a day, because I already created a post today, but I just can’t restrain myself to do it.