A RTC WorkItem Command Line Version 3.0


I was interested in how complex it might be to export work items and import them again. So I looked into this and enhanced the work item command line (WCL) to support this.

I found it quite challenging to develop this. There are a lot of things to consider, so I drove this to a point that was sufficient for my purposes. The tool is used by my team to import work items from a CSV file we receive every now and then.

I implemented another export/import mode with some more capabilities and it works in tests, but is by no means production ready. The amount of necessary tests and test automation to make this reliable, is just overwhelming. So this is not thoroughly tested.

So be careful if using these commands, and do a good amount of testing before actually using this. The problem is, there are so many possible use cases and dependencies that it is very hard to develop this kind of capability and to test it.

One special case is importing/creating links. Some links have constraints i.e. parent and especially child links. A work item can not have multiple parents. So setting child links can cause the save to fail if the new child has already a different parent.

Solution Overview

The work item command line WCL now supports two new commands

  • exportworkitems
  • importworkitems

To export work items to a CSV file and import work items from a CSV file.

I chose to use a CSV file, because RTC itself can export and import that format already. It would be ideal if export and import from XML would be supported as well, but this would require a substantial effort to abstract the export and import operations to be able to use a strategy (or some other useful pattern to support abstraction).

No Support or Maintenance

This is provided as-is with no support or guarantee. This is not a tool that is officially supported by IBM or any other organization.

Please note, that I have very little time to do this and testing is always lacking. So take the code published here with a grain of salt. On the positive side, you have the code, can debug and enhance it.

Compatibility

This code has been used with RTC 4.x and 5.x with no changes and it is pretty safe to assume, that the code will work with newer versions of RTC. The code requires two external libraries that need to be downloaded and installed separately.

License and Download

The post contains published code, so our lawyers reminded me to state that the code in this post is derived from examples from Jazz.net as well as the RTC SDK. The usage of code from that example source code is governed by this license. Therefore this code is governed by this license. I found a section relevant to source code at the and of the license. Please also remember, as stated in the disclaimer, that this code comes with the usual lack of promise or guarantee. Enjoy!

Update: Added switch to change the export and import formats for dates, see details below. I also added a switch to suppress attribute not found errors and other frequent errors during export.

Update: Fixed duration set problem. Version updated to 3.2

You can download the latest version from this post The RTC Work Item Command Line On Bluemix. The older version 3.0 can be found here:

Please note, there might be restrictions to access Dropbox and therefore the code in your company or download location.

Just Starting With Extending RTC?

If you just get started with extending Rational Team Concert, or create API based automation, start with the post Learning To Fly: Getting Started with the RTC Java API’s and follow the linked resources.

You should be able to use the following code in this environment and get your own automation or extension working. The code linked from this post contains Client API.

Setup and Usage

For the general setup follow the description in A RTC WorkItem Command Line Version 2 and look at the additional setup steps below.

For usage follow the description in A RTC WorkItem Command Line Version 2 and in A RTC WorkItem Command Line Version 2.1. Check the README.txt which is included in the downloads.

Export Work Items

The export work items command has the syntax

-exportworkitems {Switch} repository=”value” user=”value” password=”value” projectArea=”value” query=”value” exportFile=”value”  [columns=value] [encoding=value] [delimiter=value] [querysource=value]

Required Parameters are

  • repository=”value” – the repository URI, for example repository=”https://clm.example.com:9443/ccm
  • user=”value” – The user ID of the user executing the command, for example user=”ralph”
  • password=”value” – the password of the user, for example password=”password”
  • projectArea=”value” –  The project area to export items from, for example projectArea=”JKE Banking (Change Management)”
  • query=”value” – the name of the query to use, for example query=”All WorkItems”
  • exportFile=”value” – The path of the export file, for example exportFile=”C:\aaTemp\Export\Test.csv”; the folder that contains the export file must exist

Optional Parameters are

  • columns=value – The names or ID’s of the work item attributes to export; example columns=”Type,Id,Planned For,Filed Against,Description,Found In”; To specify the colums it is possible to use the name or the ID of the attribute, the switch headerIDs specifies the output format to use the ID instead of the name in the output; It is possible to use the values from an RTC Eclipse client export
  • encoding=value – The encoding; default encoding=”UTF_16LE”; options see available charset names; if the encoding is chosen different for export and import, the values will not be recognizable
  • delimiter=value – The delimiter to be used between the columns. Default is comma delimiter=”,”
  • querysource=value – If the parameter is omitted the command searches a personal query with the given name; if the value is provided a query shared by the process area is searched, a complete path from the project area to the sharing process area must be provided, for example querysource=”JKE Banking(Change Management),JKE Banking(Change Management)/Business Recovery Matters”
  • timestampFormat=value – To specify the time stamp format to be used; default “MMM d, yyyy hh:mm a”;  see SimpleDateFormat for the supported pattern

Available switches are:

  • /ignoreErrors – Ignore minor errors in mapping and value lookup
  • /asrtceclipse – Export in a format compatible to the RTC CSV export and import; if the switch is not provided, the data is exported in a format that is compatible with the syntax used by the work item command line WCL to identify elements; see A RTC WorkItem Command Line Version 2 for the supported value representations
  • /headerIDs – Export header values as attribute IDs and not as attribute names
  • /suppressAttributeExceptions – Suppresses exceptions thrown for attributes that are not available on the work item type of for attribute types that are not yet implemented

Example

-exportworkitems /ignoreErrors repository="https://clm.example.com:9443/ccm" user=ralph password=ralph projectArea="JKE Banking (Change Management)" exportFile="C:\aaTemp\Export\Test.csv" query="All" columns="workItemType,summary,Attachments"

Import Work Items

The import work items command has the syntax

-importworkitems{Switch} repository=”value” user=”value” password=”value” projectArea=”value” query=”value” importFile=”value”  [columns=value] [encoding=value] [delimiter=value] [querysource=value]

Required Parameters are

  • repository=”value” – the repository URI, for example repository=”https://clm.example.com:9443/ccm
  • user=”value” – The user ID of the user executing the command, for example user=”ralph”
  • password=”value” – the password of the user, for example password=”password”
  • projectArea=”value” –  The project area to export items from, for example projectArea=”JKE Banking (Change Management)”
  • importFile=”value” – The path of the import file, for example importFile=”C:\aaTemp\Export\Test.csv”

Optional Parameters are

  • mappingFile=”value” – A RTC work item import mapping file, for example mappingFile=”C:\temp\mapping.xml”; the file must be generated by RTC and customized to match the value mapping
  • encoding=value – The encoding; default encoding=”UTF_16LE”; options see available charset names; if the encoding is chosen different for export and import, the values will not be recognizable
  • delimiter=value – The delimiter to be used between the columns. Default is comma delimiter=”,”
  • timestampFormat=value – To specify the time stamp format to be used; default “MMM d, yyyy hh:mm a”;  see SimpleDateFormat for the supported pattern

Available switches are:

  • /ignoreErrors – Ignore minor errors in mapping and value lookup
  •  /importmultipass – Import the work items from the CSV file in a first iteration and build up a mapping for the ID’s provided in the import file and the actual ID’s created and recreate the work item links between the new work items based on that mapping in a second pass; the old work item ID for a work item has to be provided in a special column with header name com.ibm.js.oldid
  • /forcelinkcreation – if no target work item can be found in the map, use the given ID to create the link
  • /importdebug – Print more information during import attempts to help with finding issues
  • /enforceSizeLimits – Attributes such as description and medium strings have size limits, if this switch is set, the importer tries to clip content to avoid exceptions due to the size limits

Example

-importworkitems  /enforceSizeLimits  /importmultipass  /ignoreErrors repository="https://clm.example.com:9443/ccm" user=ralph password=ralph projectArea="ImportTest1" importFile=""c:\aaTemp\ExportImport\TestExportAll.csv"

RTC Eclipse Compatible Export Mode

In mode asrtceclipse, all data is exported the way RTC would export them in Eclipse. This means that certain information for example links, team areas, iterations, attachments and other data is exported in a way that makes it hard to map to data in the repository.

The import command does its best to map based on names, but for complex hierarchical information such as iterations and team areas, there is currently no search implemented that will find the object successfully. It would be possible to implement such methods, with some effort.

Example: An iteration is part of a timeline. The timeline is needed to find the iteration within. If there is no information about the timeline, it would be required to iterate all timelines with a good chance of mismatch.

The import command will try to find things by name and ID, with the limitations above.

If the work item ID attribute is provided as a column the importer will try to find the work item and update it during the import.

Default Export Mode

In the default export mode, the RTC Work Item Command line export command exports the data in greater detail, which makes it easier for the importer to identify the item.

Attachments

In default mode, the attachment is exported as a file relative to the location of the generated csv file. The attachment is downloaded to a location ./attachments//. So for each exported work item with attachments, a separate folder is created. The attachments are stored in that folder and the export information in the csv file is created compatible to the WCL parameter format to allow later import of the attachments, including upload and applying the additional information.

Other Complex Items and Links

Other complex items such as iterations and team areas are also exported with a lot more details. An iteration is being exported as path, including the timeline and parent iterations. A team area is also exported as path containing the Project are name and parent team area names

Multi-Pass Import

Importing work items and recreating the link relationships between them is problematic, because while importing the work items the link target may not yet exist. To be able to import a set of work items and then recreate the linkage, it is necessary to do the import and then map the ID of the old work item to the ID of the new work item.

When using the RTC CSV importer in the Eclipse client, existing work items are provided with a # in front of the work item ID. To do an import and then recreate the links between the new work items (and not to the old ones in the import), a user would have to run the import without the links, then replace the work item ID’s in the import file by the new work item ID’s and update the work items with a second import. This is very manual and error prone.

The switch importmultipass  enables an import mode, where the WCL tries to create the links between the imported work items, rather to the old ones. It imports the work items in two passes. It creates the work items in the first pass and ignores the link creation. In the second pass it tries to create the links. For links between work items WCL tries to find the work items that were created during the import and tries to match the links to the new work items, where possible.

Note: Only links between work items are handled this way. Links to objects other than work items are recreated using the values provided in the import file.

To be able to do this, the import file has to provide the old work item ID of the work items that are imported. The import requires a special ID for the columns containing the old ID’s. The column header for this column has to be specified with com.ibm.js.oldid.

The import file below has been created using an export that included the ID of the work item in the export. The old column header for example ID of the column has been replaced by com.ibm.js.oldid. The work item links show the ID’s of the linked work items with their old ID’s.

Import Work Items With LinksThe import works as follows.

The WCL runs the first pass and imports the work items. It stores the mapping between the original work item ID from the column com.ibm.js.oldid and the ID of the newly created work item in a map. Links are not created in this pass.

In the second pass WCL reads the import file again and only handles the columns that represent links. It detects if the link target represents a work item. If not, it tries to recreate the link as it is. If the link is a work item link, WCL tries to calculate if a new work item was created for the target using the map. If the work item was imported and a new ID is available, the new work item ID is used to create the link.

If the ID of the link target can not be found in the mapping, WCL can either ignore the link or it can try to create the link to the original work item. WCL supports these two modes. By default, the link is not created. If the switch forcelinkcreation is specified, the original value of the target work item ID is used as target for the link, if no mapping to a newly imported item was found.

Creating links is not trivial. One special case is importing/creating links. Some links have constraints i.e. parent and especially child links. A work item can not have multiple parents. So setting child links can cause the save to fail if the new child has already a different parent. This can create issues in import scenarios, especially if an export from the same repository is imported and the import causes child links to be created that have already another parent. In this case the import will fail with an error.

Limitations

Approvals and comments are imported into one comment. The effort to recreate approvals is just too big and I can’t see the added value.

Special Notes On Setup

For the general setup follow the description in A RTC WorkItem Command Line Version 2 and look at the additional setup steps below.

The export and the import commands of WCL need two libraries that are not shipped the downloads.

If you use the packaged WCL and want to use the export/import capability follow the steps below to add the required libraries to the folder lib in the folder lib in the WCL folder.

If you use the Eclipse project for WCL and want to use the export/import capability follow the steps below to add the required libraries to the folder lib in the Eclipse project com.ibm.js.team.workitem.commandline.

The export and the import commands of WCL use the Open CSV Library. I had issues with the newer versions of Open CSV that I could not resolve, so this code assumes the version 2.3. Download the version 2.3 from here. Uncompress and untar the the file opencsv-2.3-src-with-libs.tar.gz you downloaded. Look for the folder opencsv-2.3\deploy\ copy the JAR file opencsv-2.3.jar and put it into the lib folder of your version of WCL.

The import commands of WCL can only provide the capability to use a mapping file by using a JAR file that only ships with the RTC Eclipse client and the SDK. The classes used for the mapping file capability are located in the library com.ibm.team.workitem.rcp.core.  Open the Install location of the RTC Eclipse client and search for com.ibm.team.workitem.rcp.core*. You should find a file names similar to this one: com.ibm.team.workitem.rcp.core_3.1.900.v20141010_0124.jar. The version numbers at the end could be different. Copy the JAR file into the into the lib folder of your version of WCL.

The packaged version should look like below.

Deployed Packaged WCLIf you have imported the Eclipse project for WCL open Configure Build Path and create a user library named openCSV and add the Open CSV library opencsv-2.3.jar. Create a user library named rtcmapping and add the com.ibm.team.workitem.rcp.core library you just copied to it.

Your Eclipse Project should now look like below.

Eclipse Project and LibrariesCode Changes

During the work on import and export, the code structure was left untouched. Some classes were added to be able to handle the column header and some additional mapping of id’s and names i.e. for link types. In addition some of the code that was piling up in the WorkItemUpdateHelper (formerly known as WorkItemHelper) was moved to utility classes. this makes it also easier to look for useful API in case you are interested in how things work in the RTC API. See the scree shot below.

Code StuctureSummary

This WorkItem Command Line should allow for most of the automation needs when creating work items. In addition it is a nice resource for the RTC work Item API.

As always, I hope the post is an inspiration and helps someone out there to save some time. If you are just starting to explore extending RTC, please have a look at the hints in the other posts in this blog on how to get started.

106 thoughts on “A RTC WorkItem Command Line Version 3.0

  1. Great tool Ralph. I took a shortcut and just installed it (didn’t build it with the code). It was easy to install and set up. Only one issue…. it wants to see all timestamps as MM d, yyy hh:mm a, and my test file had them as MM/DD/YYYY. Any idea how to fix this?

    • Timestamps are an ugly bunch, Dan. There is tons of encoding and decoding going on. I hardcoded the format. Look into com.ibm.js.team.workitem.commandline.utils.SimpleDateFormatUtil and choose your encoding.

    • Unfortunately there are other places where the string is used. I will try to take some time to look at it. For now, you will have to look where it is used, unfortunately and enter your own pattern there.

  2. Thanks for the fabulous tool! Very excited to start using it to drive some automation in our fast-paced setting!
    …however, having some trouble with getting durations to create/update properly (the duration, timeSpent, and correctedEstimate fields)…they show up as coming through, but always to go to “0 hours”, regardless of the millis I’m inputting…i.e. 3600000.
    Any thoughts as to how I can get this to update properly?

    • Paul, can you provide me with how you call WCL? The command and parameters?
      Can you tell me the attribute ID and type and how it is configured in the presentation?
      I did a lot of changes for the export and import part. Maybe I overlooked something.

      You can also debug it yourself. The code relevant should be in com.ibm.js.team.workitem.commandline.helper.WorkItemUpdateHelper.getRepresentation(ParameterValue, List). Look for “duration”.
      The call is getDurationFromString(). It is really a very simple call. Not sure how anything should go wrong there. But software,…… anything can go wrong.
      I would try to debug it next week, if you can provide me with the information above.

      • I see the attributes you mean. I took a peek and found the error. I think I have a fix.
        All these test to whatever conversions are a nightmare, to be honest. I will ship a fix next Monday.

      • com.ibm.js.team.workitem.commandline.utils.SimpleDateFormatUtil.convertDurationToMiliseconds(String)

        Needs to be changed to:
        /**
        * Takes a duration and converts it into a long containing the miliseconds.
        * Input format: 3 hours 3 minutes
        *
        * @param duration
        * @return
        */
        public static Long convertDurationToMiliseconds(String duration) {
        long time = 0;
        if (duration == null) {
        return time;
        }
        int hoursIndex = duration.indexOf(DURATION_HOURS);
        int minsIndex = duration.indexOf(DURATION_MINUTES);
        if (hoursIndex < 0 && minsIndex 0) {
        String hours = duration.substring(0, hoursIndex – 1).trim();
        time += TimeUnit.HOURS.toMillis(new Long(hours));
        }
        if (minsIndex > 0) {
        int start = 0;
        if (hoursIndex>0){
        start = hoursIndex + DURATION_HOURS.length();
        }
        String minutes = duration.substring(
        start,
        duration.length() – DURATION_MINUTES.length()).trim();
        time += TimeUnit.MINUTES.toMillis(new Long(minutes));
        }
        return time;
        }

      • Hey, it works perfectly! Thanks very much for the quick fix, much appreciated! …and I like the hours/minutes enhacement…very nice! 🙂

      • Thanks. I will have a look if I can replace my own code with something based on a standard library in the future. This might make things easier.

  3. Hi
    I’m in trouble with exportworkitems.

    The error message says that the server address is not available, but the repository url it is correct and server is avaible.

    Any idea ?

    c:\IBM\jazz\WCL>”C:\Program Files\IBM\TeamConcert\jdk”\jre\bin\java -Djava.security.policy=rmi_no.policy -Djava.ext.dirs=./lib;C:\IBM\jazz\WCL-Hooks\wcl\PlainJavaAPI;”C:\Program Files\IBM\TeamConcert\jdk”/jre/lib/ext -cp ./lib;C:\IBM\jazz\W
    CL-Hooks\wcl\PlainJavaAPI -jar wcl.jar -exportworkitems /ignoreErrors repository=”https://mysrvurl:9443/ccm” user=”my_user” password=”*****” projectArea=”FrontBox CCM” exportFile=”C:\IBM\export.csv” query=”etl” columns=”summary”WorkItemCommandLine Version V3.2

    Starting Team Platform …
    Executing command: exportworkitems

    Exception in thread “main” java.lang.IllegalArgumentException: The server address is not valid.
    at com.ibm.team.repository.client.internal.TeamRepository.(TeamRepository.java:382)
    at com.ibm.team.repository.client.internal.TeamRepositoryService.createSharedTeamRepository(TeamRepositoryService.java:526)
    at com.ibm.team.repository.client.internal.TeamRepositoryService.getTeamRepository(TeamRepositoryService.java:113)
    at com.ibm.team.repository.client.internal.TeamRepositoryService.getTeamRepository(TeamRepositoryService.java:132)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepositoryCommand.login(AbstractTeamRepositoryCommand.java:147)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepositoryCommand.execute(AbstractTeamRepositoryCommand.java:81)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.runCommands(WorkitemCommandLine.java:293)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.run(WorkitemCommandLine.java:224)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.main(WorkitemCommandLine.java:174)

  4. Hi Ralph, my last message was because I ran the program like this:
    ./wcl.sh -exportworkitems /ignoreErrors repository=”https://server.sby.ibm.com:9445/ccm” user=*** password=*** projectArea=”SCM RTC POC” exportFile=”/tmp/export_pa_tst.csv” query=”All” columns=”workItemType,summary,Attachments”

    and I get this error:
    Executing command: exportworkitems

    Exception in thread “main” java.lang.NoClassDefFoundError: au.com.bytecode.opencsv.CSVWriter
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.createWriter(ExportWorkItemsCommand.java:376)
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.process(ExportWorkItemsCommand.java:311)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepositoryCommand.execute(AbstractTeamRepositoryCommand.java:89)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.runCommands(WorkitemCommandLine.java:293)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.run(WorkitemCommandLine.java:224)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.main(WorkitemCommandLine.java:174)
    Caused by: java.lang.ClassNotFoundException: au.com.bytecode.opencsv.CSVWriter
    at java.net.URLClassLoader.findClass(URLClassLoader.java:434)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:672)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:358)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:638)
    … 6 more

  5. Hi Ralph, I did all setup as provided and I get the below error while executing the “exportworkitems” command. There are no underscores or special characters in the server name.
    Note: The other commands like printtypeattributes & update works just fine.

    C:\RTC502Dev\RTCWCL\WCL>wcl -exportworkitems /ignoreErrors repository=”URI” user=….. password=….. projectAre
    a=”Global Financing Portfolio” exportFile=”C:\RTCWCL\trekexport.csv” query=”TREK
    Work”

    C:\RTC502Dev\RTCWCL\WCL>set JAVA_HOME=C:\RTC502Dev\installs\TeamConcert\jazz\cli
    ent\eclipse\jdk

    C:\RTC502Dev\RTCWCL\WCL>set PLAIN_JAVA=C:\RTC502Dev\installs\PlainJavaAPI

    C:\RTC502Dev\RTCWCL\WCL>C:\RTC502Dev\installs\TeamConcert\jazz\client\eclipse\jd
    k\jre\bin\java -Djava.security.policy=rmi_no.policy -Djava.ext.dirs=./lib;C:\RTC
    502Dev\installs\PlainJavaAPI;C:\RTC502Dev\installs\TeamConcert\jazz\client\eclip
    se\jdk/jre/lib/ext -cp ./lib;C:\RTC502Dev\installs\PlainJavaAPI -jar wcl.jar -ex
    portworkitems /ignoreErrors repository=”https://igartc01.swg.usma.ibm.com” user=
    ….. password=….. projectArea=”Global Financing Portfolio”
    exportFile=”C:\RTCWCL\trekexport.csv” query=”TREKWork”
    WorkItemCommandLine Version V3.2

    Starting Team Platform …
    Executing command: exportworkitems

    Exception in thread “main” java.lang.IllegalArgumentException: The server addres
    s is not valid.
    at com.ibm.team.repository.client.internal.TeamRepository.(TeamRep
    ository.java:382)
    at com.ibm.team.repository.client.internal.TeamRepositoryService.createS
    haredTeamRepository(TeamRepositoryService.java:526)
    at com.ibm.team.repository.client.internal.TeamRepositoryService.getTeam
    Repository(TeamRepositoryService.java:113)
    at com.ibm.team.repository.client.internal.TeamRepositoryService.getTeam
    Repository(TeamRepositoryService.java:132)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepository
    Command.login(AbstractTeamRepositoryCommand.java:147)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepository
    Command.execute(AbstractTeamRepositoryCommand.java:81)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.runCommands(
    WorkitemCommandLine.java:293)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.run(Workitem
    CommandLine.java:224)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.main(Workite
    mCommandLine.java:174)

  6. Hi Ralph.
    I have a question regarding extensibility of the tool. The tool really works well. Only one (hopefully small) piece is missing (from my point of view).
    In the current state the tool takes a predefined query as input for reading the work items. Is there any chance to make the input definition exchangeable?
    My scenario is that we often need to copy work item hierarchies (that cannot be described using a query). For that we have a library that can traverse work item hierarchies. It would be ideal if I could use my traversal library to ‘provide’ your tool the work item instances for further processing.

    Please think about this “enhancement request” and give me your feedback.

    Thank you

    • Marko, the tool is provided as is, with all the code. I have no plans currently to enhance and change the tool. I basically don’t have the bandwidth to do all the tests.

      If you want to enhance the tool to work in a different fashion, go ahead and do it. You can also look at the export/import feature that I have created in the last version as a demonstrator/prototype, how to extend it for you usage. The import code basically parses the external data and creates the parameter list that is later used to update/create the work items. This is a valid strategy that you could potentially also use with the tool you already have.

  7. Hi Ralph,
    Great tool. thank you.
    I am moving records from blue mix RTC to RTC in our organization. I like to change Work item type of “Epic” to “Program Epic”. To do this, I simply edited the cvs file I exported from bluemix and changed the word for type and tried to import the CSV file. I noticed the import failed with failure such as below.
    Reading row 1 : [Type, Summary, Status, Owned By, Priority, Severity]
    Reading row 2 : [牐杯慲Epic, Server Onboarding Process, New, Unassigned, Unassigned, Normal]
    Mapped Attribute: [workItemType] Mapped Value: [牐杯慲Epic]
    I used simple text editor to update the cvs file, however as you can see it has these special characters.

    Is it allowed to edit the cvs file?

    • If you export/import you do that with a certain encoding. You have to make sure you keep that encoding or change it to what you want. Otherwise the importer can’t read the data and you see funny characters. Use notepad++ or another editor where you can actually control the encoding.

  8. Hi Ralph,

    If I try to import a WI that references more than one attachement I get the issue like:

    Starting Team Platform …
    Executing command: importworkitems

    …Pass: 0
    Exception! Attach File – File not found: null\Attachments/65839/capture_1468499609.png
    Ignored…….
    Created work item 66044.

    Success!

    —————————————————————————————-
    The attachement capture_1469019314.png is imported, the second attachement(capture_1468499609.png) could not be imported.

    Import file:
    “workItemType”,”summary”,”category”,”internalTags”,”description”,”Attachments”
    “Task”,”Summary_839″,”PCPS_Test”,”wcl_test_1, DestinationTag”,”Description in 65839Zweite Zeile in 65839″,”D:/Users/Public/RCP_Int/Workspace_RCP_int/Phyton_1/src/Attachments/65839/capture_1469019314.png,capture_1469019314.png,image/png,UTF-8
    Attachments/65839/capture_1468499609.png,capture_1468499609.png,image/png,UTF-8″

    Thank you

    • Dietmar, this code is basically experimental. I tried to run tests, but obviously this is not product quality code. You would have to setup a development environment and fix the problem.

      I am unfortunately too busy right now to do anything here.

  9. If I am executing a script via a JCL/Batch job to a unix script to run the utility, can I point the output to a unix location rather than a PC location for the csv file ?

    Or rather just do the query and write the output to the log ?

    I use this method for creating/updating/closing Work Items for mainframe automation. But wanted to be able to run a query to see what work items still need to be processed and send out reminders based on internal work rules/criteria.

    So I was basically trying to just get a report based on the query specified, or even a dump to the log of the CSV information if I can not store it in a unix folder location.

    Thanks!

    • Joe, I developed it as an example and tested it for my needs. I don’t have the resources to also perform cross platform development and test.

      I don’t do any checks for the file name you pass. It is only a string until it is passed ahead.

      But if anything fails, the code is shipped, so you can customize it to your needs.

      • Before I start digging into the code, may I ask if there a method already in the tool to do the query like the exportworkitems and just dump the output to the standard output, rather than writing it to a file ?

        Thanks in advance,
        Joe

  10. Hi Ralph,

    thank you very much for publishing your tool. It helps me a lot in what I am currently doing.

    During some tests, I found two issues that you might be interested in (if you continue expanding the code base):

    1. Writing contributors whose name contains a comma (“,”) e.g. to a meeting work item’s participants attribute is not possible. I debugged into your code and found that there is an ITEM_SEPARATOR constant in the WorkItemUpdateHelper.java. This constant being set to “,” seems to be causing this kind of trouble. For me, this is critical as I have lots of users with a comma in the user name due to some naming pattern.

    2. Exporting the internalResolution attribute of a work item writes something like
    “com.xyz.rtc.configuration.workflow.workflowdefinition.defect.resolution.r9” to the CSV. Importing this back to RTC results in a “Resolution not found” kind of exception. Anyhow, the import succeeds after changing the resolution in the CSV manually to the display name (e.g. “resolved” or “invalid” instead of the lengthy term mentioned before).

    It would be really great if you could consider those two issues once you provide any kind of update to the tool. The reason is that I do not want to change too much of your code as I am somehow afraid of merging once you provide an update bringing up some new features.

    • Richard, I have no plans at the moment to put more work into the tool. Please note that the ITEM_SEPARATOR is used as separator for all item list attributes like userList. I had to pick some choices and never had the problem you have. You should be able to consistently change it.

      The resolution, I have tried to search for name and ID, maybe there is nothing built in for resolutions. If you fix it, consider to share the code changes back with me.

      • Hi Ralph,
        thank you for your quick reply. I set up some development project and might continue modifying WCL. As of now, handling the “resolution” issue within a component outside WCL is easier for us.
        As the ITEM_SEPARATOR was most critical to me, I changed it to the null character ‘\u0000’. This works fine and resolves the issue for all of our use cases.
        By the way, I came across one more issue with “/importmultipass”. As I am dealing with a highly customized setting, we have custom work item attributes of type “workItem” (which are not links). Work items with those attributes are not considered by the second pass. This is causing me some trouble in particular use cases. Might be that I will go for extending WCL so that a case like this is handled as well.
        In case I decide to do any more modifications to WCL, I will provide you with a patch so that you can re-share the fixes with the community.

  11. Hi Ralph ,

    I wonder if you know of any possibility to import work items in more than one project area at the same time .

    At . ,

      • We are working with the sharing process, with 500 project areas linked to an area to prove the process.
        We have the need to apply a mask to the attribute , only that there are several values ​​that are not compatible.
        I would export these values​​, update, and update them on workitens .
        This is needed.

      • Well, the code is downloadable on the post, so you can make anything you want out of it. I don’t have the bandwidth to code solutions for special requirements, unfortunately.

  12. Hello Ralph,
    The IBM consultant who is helping us to introduce RTC in our company referred to your tool which looks really helpful for our migration from another system.
    I found out, that the importworkitems function can also be used to just “update” items, if they already exist in the system: All attributes specified will be changed, all others will be left as-is. This is a helpful feature during our transition phase when work is still done in the old system.
    However: I cannot update the attachments this way. Even when specifying the same filename and description, the file is uploaded again (and I have now 2 attachments).
    When using the @attachFile parameter for single item update, this works flawless. So is this just missing for the -importworkitems command or did I miss sth.?
    Or is there an alternative way of removing all attachments (of multiple items) and then running the import procedure?

    Best regards
    Ruben

    • Ruben,

      you have to understand that this is a pet project and the intent is to provide users with the code to show how the API works. I don’t have the time to have this as a production tested tool. The Import tool basically uses the UpdateWorkItem command and passes @attachFile with the file name in the default mode. The simple assumption being that you likely want to upload attachments and not remove the old ones. You would have to set the switch enableDeleteAttachment in com.ibm.js.team.workitem.commandline.commands.ImportWorkItemsCommand.process() and set the mode set in com.ibm.js.team.workitem.commandline.commands.ImportWorkItemsCommand.addAttachmenParameter(ParameterList, String, String).

      So you could add this functionality if you want. It was just not in the scope of what I had in mind/needed when I created this code.

  13. Hi Ralph,
    can you tell me the importance/reason for the MIME type which is the 3rd parameter within the attachment column?
    When manuall uploading a file with the web GUI, RTC sets some MIME type. I can see this by doing an export, which for example gives me “application/vnd.ms-excel” as MIME type for a csv file:
    “Id”,”Attachments”
    “2312”,”./Attachments/2312/csvFile.csv,csvFile.csv,application/vnd.ms-excel,UTF-8

    So RTC magically determines the MIME type based on the file suffix or sth. else.
    However, as we try to migrate data from a system into RTC, we don’t know the RTC-internal MIME type for all types of documents/pictures/etc. which must be set when uploading with the JAVA API.

    I simply uploaded an excel file with the MIME type set to “application/unknown” and it works (of course).
    I guess the MIME type could somehow be used for an attachment preview within the webbrowser (which is not working in my browser anyway).
    So am I disabling any future preview feature when uploading all our attachments with MIME type “application/unknown”?

    • I have no clue how that really works. But apparently this is used in SCM systems to determine between dealing with binaries or text. almost everything I see here is application binary. Only Text documents are “plain/text”. I think the Web UI provides a preview for “plain/text”. But, to be honest, I can’t suggest you too much here. It is a bit mysterious in Eclipse it is specified in the Preferences Team>Jazz Source Control>File Properties

      • Unfortunately, the Web UI does not provide a preview for “plain/text”, nor any other MIME type I tried, with one exception:
        After uploading a jpeg image (the MIME type was set to “image/jpeg” automatically), I can preview the picture in the browser when clicking on the attachment without needing to download it first. Nice, but still mysterious. I could not find any more helpul info in this on jazz.net either.
        Maybe to jpeg preview is the template preview feature an extemsion of this is at least forseen (if not planned) by your colleagues that are developing the web UI.

      • Too much to know it all. I know it shows images but I don’t have the time to test. And to be honest, if it is a huge file it also takes a while to load it to show it. So maybe it is better to limit what you preview.

  14. Hi Ralph, may I know this WCL can work with IBM RTC DevOps, it seems I put my credential correctly but it giving me below error

    WorkItemCommandLine Version V3.2

    Starting Team Platform …
    Executing command: exportworkitems

    Sep 11, 2016 9:40:29 PM org.apache.commons.httpclient.auth.AuthChallengeProcesso
    r selectAuthScheme
    INFO: basic authentication scheme selected
    Sep 11, 2016 9:40:31 PM org.apache.commons.httpclient.auth.AuthChallengeProcesso
    r selectAuthScheme
    INFO: basic authentication scheme selected
    Sep 11, 2016 9:40:34 PM org.apache.commons.httpclient.auth.AuthChallengeProcesso
    r selectAuthScheme
    INFO: basic authentication scheme selected
    TeamRepositoryException: Unable to log into repository!
    CRJAZ0105E The request for the following URL was denied: /ccm[maskvalue]/jauth-issue-toke
    n. The status is “Unauthorized”.

    Failed!

      • Hi Ralph sorry late reply, I able to login and access the RTC DevOps, just a simple typo and workaround about the prjct area value.

  15. Hello Ralph. We are using the tool for doing work item data migration after we have changed e.g. some states in the process configuration. Now when we run the tool each of the work item changes leads to an email. This is confusing the normal end users that have not touched their work items at all but instead this was done by the “data migrator” using a script.
    Would it be possible to introduce an additional command line parameter for skipping email notification. In the API this can be easily realized by adding an IAdditionalSaveParameters constant to the working copy, see https://jazz.net/wiki/bin/view/Main/SkipMailSaveParameter

    This would really help us a lot.

    BTW. Do you plan to make this tool completely open source and give somebody in addition to yourself write permissions to the sources?

    • Hi Marco,

      The tool comes as source code, so you can add your own save parameters to it. There is actually an example when creating links with backlinks in the code.

      The source code is available, feel free to customize it as you please. I had to put the license in there that was in the IBM Examples. I can’t make it open source. I don’t know what would be required to do so, but I ship the code under the assumption that users use and change it.

      • PS, adding the parameter only works for RTC 6.x, so it should be in a special version. The rest of the API should work for almost all versions of RTC.

    • You would have to

      * Use the work item import and a CSV file
      * Call WCL for each work item ID separately (consider using the RMI option)
      * Extend WCL to accept multiple ID’s

      It can be done.

    • Is there any specific reason why you want to use the -update command instead the -importworkitems command which is much more convenient in your situation (change multiple items)?

      • Users have requested on Jazz.net to be able to for example update multiple work items e.g. add a tag to all of them. It would be easier to do something like that if you would be able to call it directly somehow and not having to create a CSV file.

        It is doable, but would require enhancing the code.

  16. Hi Ralph,
    is it also possible to “remove” links using the importWorkItems command?
    I could only find the remove mode for single work item related command line calls.
    Thank you

  17. Hi ralph
    I’m using wcl 3.2 and open-csv 2.3 and finnaly got the libraries worked (BeforeI tryed with wcl 4 and open csv 2.3 or 3.7 neither worked).

    But now I’m getting an error that says :
    “Exception creating output writer!
    Failed!”

    Looking at the wcl code that downloaded apart, it seems that this messages goes when this 2 things happens :
    FileNotFoundException or UnsupportedEncodingException . Any hint with this?

    Folder of my code exist. Does the file test.csv has to exist? I tried creating it before or just leaving the folder empty.

    This is my command :
    wcl -exportworkitems /ignoreErrors repository=”https://****/ccm” user=administrador password=**** projectArea=”GIT” exportFile=”C:\RTCWCL\test.csv” query=”All” columns=”workItemType,summary”
    Thanks

  18. Hi Ralph

    doing some debugging I added in the cause to the error message:

    } catch (UnsupportedEncodingException e) {
    throw new WorkItemCommandLineException(
    “Exception creating output writer! UnsupportedEncodingException”, e);
    } catch (FileNotFoundException e) {
    throw new WorkItemCommandLineException(
    “Exception creating output writer! FileNotFoundException”, e);

    And it seems it is “FileNotFoundException”. What did does mean? THe folder is created already.

    • FileNotFoundException is usually thrown if the file one tries to access is not found. e.g. if the csv file expected is not found or whatever file that is used in the call.

  19. Hi Ralph
    I’m facing a problem when 2 attachments are named the same (It happens regularly in the Project Areas I’m working). While looking at the ExportWorkItemsCommand.class, I can not figure out if I attachment name can be the attachment id + name because when 2 attachments are named the same it replaces the 1st one.

    Regards

    • Cesar,

      I did not code for such a case. You could add this to com.ibm.js.team.workitem.commandline.utils.AttachmentUtil.saveAttachmentToDisk(IAttachment, File, IProgressMonitor). You would basically have to create a unique file name e.g. add some id. Obviously you would have to add some special handling in the import work items to handle the attachment name.

  20. Hi Ralph,
    WCL just helped me save a lot of time by creating a lot of approvals. Is it also possible to set an approval to “approved”? I could not find a hint to this in the help.
    Thanks

  21. hello ralph

    I can run the export command successfully. But I want to run the query with condition.
    How to add the condition in the query like “Field Against=A” when export command is executed?

  22. Hello,I meet a issue.Could you help me if you have related experience.
    When run the command , I used to meet this error:
    -bash: users/xxxx/downloads/rtc502dev/TeamConcert/jazz/client/eclipse/jdk/jre/bin/java: No such file or directory
    or
    Unable to launch “users/xxxx/downloads/rtc502dev/TeamConcert/jazz/client/eclipse/jdk/bin/java” (8)

    why did these happen.Could you give some advise.
    Thanks and best regards.

      • Thank you for your answer. I’ll try it . By the way ,I am working on Mac . Could this tool work on macOS.

      • I asked but there was no offer to donate me a Mac, hence I can’t test that.

        1. I have not heard any complaints from MAC users.
        2. My requirement is Java and the Plain Java Client libraries. As long as the Eclipse client runs in the environment, the WCL should.
        3. You might have to rebuild and repackage with the Java version you use. See ‘ReadMe – HowToRelease.txt’ in the project root.

  23. hi ralph
    I know the reason why I could not run the wcl when the query that created by the other people.
    It is necessary to add querysource. Sorry for disturbing you. Please forget it.

    • WCL needs to know where to search the query and WCL also must have permission to see and execute the query. E.g. if you run as a normal user and want to pick a query that is not shared with that user, you won’t be able to run that.

  24. Hi Ralph,
    Great work.. Really helps a lot of people, especially while performing the migration tasks.

    I have a question. Do you also have something for RQM aswell.
    At the moment, I have downloaded the Test Case Results in the XML format using REST APIs and I have parsed them using Python. But, I am having problems with Attachments which are attached to the Test Steps when they fail. Through REST APIs I even was able to successfully get the integration links of the attachments and it works if I enter it in the browser one by one (it downloads the attachments in the downloads folder), but when I download these attachments through wget programatically, its downloading some unreadable file.
    Not sure if this is the right method I am following. If you have any other scripts you have developed, please share it with us.

    • Hi,

      since it is a completely different API in the QM areas, I have not done much for this. I share all I can share, so no, I don’t have anything. Have you looked at the Extra tools like the Rational Quality Manager Copy Utility and the Rational Quality Manager URL Utility?

      • Hello Ralph,
        Apologies for the delayed reply, I was not available in office until today .
        Yes, I did explore those options and hence I was able to successfully download the test case results in the XML format using RQMURLUtility.
        I even parsed the XML using XPaths with the help of Python. Now the only problem is with downloading the attachments from the test case result steps (usually testers would have attached a screenshot as to which step the test case failed and this step will usually have the defect also created).
        As you haven’t explored the RQM side of the CLM from the API/Scripting point of view, I will not disturb you more on this topic.
        But again, you have made a wonderful utility in RTC for downloading attachments. Thanks again for that.
        BR
        Venkatesh Prasad

  25. Hi Ralph, Thank you for great tool!
    I tried to export workitems with attachments but got the following error. Could you help me?
    I can execute this command succesfully if I remove “Attachments” from the parameter “columns=”.
    TIA.

    C:\IBM\TeamConcert\RTCWCL\WCL>..\..\jdk\jre\bin\java -Djava.security.policy=rmi_no.policy -Djava.ext.dirs=./lib;../../RTC-Client-plainJavaLib-5.0.1;..\..\jdk/jre/lib/ext -cp ./lib;../../RTC-Client-plainJavaLib-5.0.1 -jar wcl.jar /ignoreErrors -exportworkitems repository=”https://localhost:9443/ccm” user=bob password=bob projectArea=”JKE Banking” query=”all_with_attachments” exportFile=”wclout_attachments.csv” columns=”workItemType,summary,Attachments”
    WorkItemCommandLine Version V3.2

    Starting Team Platform …
    Executing command: exportworkitems

    Exporting work item 40
    Exception in thread “main” java.lang.NullPointerException
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.calculateAttachmentsAsString(ExportWorkItemsCommand.java:642)
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.getStringRepresentation(ExportWorkItemsCommand.java:466)
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.getRow(ExportWorkItemsCommand.java:408)
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.process(ExportWorkItemsCommand.java:349)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepositoryCommand.execute(AbstractTeamRepositoryCommand.java:89)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.runCommands(WorkitemCommandLine.java:293)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.run(WorkitemCommandLine.java:224)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.main(WorkitemCommandLine.java:174)

    • Version 3 is outdated, use a newer version if possible. You need to provide an absolute path to a output folder that actually exists.There is source code available you could look into. The null pointer happens when the filename to store the attachment is calculated.

  26. Hello Ralph,

    Even after installing the supporting libraries I still get the following error.

    C:\RTCWCL\wcl>wcl -exportworkitems /ignoreErrors repository=”https://localhost:9443/ccm” user=8888 password=8888 projectArea=”SOM” exportFile=”C:\Export_WI_With_Attachments_Test.csv” query=”All” columns=”workItemType,summary,Attachments”

    C:\RTCWCL\wcl>set JAVA_HOME=C:\RTCWCL\TeamConcert\jazz\jdk

    C:\RTCWCL\wcl>set PLAIN_JAVA=C:\RTCWCL\PlainJavaAPI

    C:\RTCWCL\wcl>C:\RTCWCL\TeamConcert\jazz\jdk\jre\bin\java -Djava.security.policy=rmi_no.policy -Djava.ext.dirs=./lib;C:\RTCWCL\PlainJavaAPI;C:\RTCWCL\TeamConcert\jazz\jdk/jre/lib/ext -cp ./lib;C:\RTCWCL\PlainJavaAPI -jar wcl.jar -exportworkitems /ignoreErrors repository=”https://localhost:9443/ccm” user=8888 password=8888 projectArea=”SOM” exportFile=”C:\Export_WI_With_Attachments_Test.csv” query=”All” columns=”workItemType,summary,Attachments”
    WorkItemCommandLine Version V4.0

    Starting Team Platform …
    log4j:WARN No appenders could be found for logger (com.ibm.team.repository.common.internal.util.InternalTeamPlatform).
    log4j:WARN Please initialize the log4j system properly.
    Executing command: exportworkitems

    Exception in thread “main” java.lang.NoClassDefFoundError: com.opencsv.CSVWriter
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.createWriter(ExportWorkItemsCommand.java:375)
    at com.ibm.js.team.workitem.commandline.commands.ExportWorkItemsCommand.process(ExportWorkItemsCommand.java:310)
    at com.ibm.js.team.workitem.commandline.framework.AbstractTeamRepositoryCommand.execute(AbstractTeamRepositoryCommand.java:89)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.runCommands(WorkitemCommandLine.java:293)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.run(WorkitemCommandLine.java:224)
    at com.ibm.js.team.workitem.commandline.WorkitemCommandLine.main(WorkitemCommandLine.java:174)
    Caused by: java.lang.ClassNotFoundException: com.opencsv.CSVWriter
    at java.net.URLClassLoader.findClass(URLClassLoader.java:602)
    at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:777)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:750)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:731)
    … 6 more

  27. HI Rallp, I use the rtcwi list command since several years now and had no issue so far also when the SCM server version changed to version 6.0.x but with the update to 6.0.6.1 I get an error stating “RJ02010: Unable to list queries”. Do oyu know if the API changed with this version ?

  28. Hi Ralph,
    Sometimes a Work Item is associated with Links or Process Links such as Parent, Children, Related, etc. I wonder how to export those links as well?
    Thank you in advance

  29. Hi Rallp im trying to get the all attributes from the workitem and im using Server sdk could you please let me know from where to get the details of RTC APIs
    Thanks & Regards
    Ayaan

Leave a reply to Marko Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.