Can’t connect to RTC running with Jazz Authentication Server


Are the Eclipse client and RTC 6.0.x Plain Java Client Library based automation and tools no longer working since RTC was set up with the Jazz authentication Server? This could be due to an issue with the shipped codecs.

There is apparently an issue with Java API clients in environments where CLM, especially the RTC Server is set up with the Jazz Authentication server. The Eclipse client and Plain Java Client Library based automation can not connect to RTC any more. See the related RTC RFE 371953.

The reason seems to be due to the codecs shipped with the Eclipse client and the plain Java Client libraries. Please see the post RTC Plain Java API and Jazz Authentication Server.

Summary

The issue seems to be:

  1.  Jazz Authentication Server requires at least version 1.4 of the Apache commons-codec library.
  2. The RTC Java client includes version 1.3.  Replacing 1.3 with 1.4 or higher resolves the issue.

 

General Solution

In the Eclipse client and the Plain Java Client Libraries folders, find and replace the file org.apache.commons.codec_1.3.0.v20100518-1140.jar with a 1.4 or later.  Version 1.10 is the latest  you can get it from https://commons.apache.org/proper/commons-codec/download_codec.cgi all prior versions are here:  http://archive.apache.org/dist/commons/codec/binaries.  Then update your classpath to point to the new .jar.

How to implement this, differs for the several options like Eclipse Client for a user, Eclipse Client for SDK and Plain Java development and so forth. If you create extensions and automation, my suggestion is to develop against a Jetty based test server and a real development Apache or WAS Liberty Profile based system without the  Jazz Authentication Server. If you want to use the automation against a life server, you can follow the description below to get your final automation up and running.

I will try to find the time to describe more details for the Eclipse client itself, but would suggest to get in contact with IBM support in the mean time.

Solution for Plain Java Client Libraries

I was able to test this today and got it working for a Plain Java Client Libraries application that was running stand alone outside of the Eclipse client. So far I tested RTC 6.0.2.

This did not work for me with RTC6.0 against an RTC 6.0.3 M6.

I downloaded the file commons-codec-1.10-bin.zip from https://commons.apache.org/proper/commons-codec/download_codec.cgi and extracted the content. In the extracted file I located the commons-codec-1.10.jar file.

I created a folder ApacheCommons in the same folder that contains my Plain Java Client Libraries JAR file and placed a copy of the commons-codec-1.10.jar there.

The application is in a file application.jar. I use a batch file to start it. I just added the folder ApacheCommons to the class path as first entry. This way the first commons codec that is found is the newest one. When running the batch file the plain java client libraries application ran as expected.

The batch file looks as follows.

set JAVA_HOME=..\TeamConcert\jazz\client\eclipse\jdk
set PLAIN_JAVA=../PlainJavaAPI
set COMMONS=../ApacheCommons
set REPOSITORY="https://exampe.com:9443/ccm/"
set USERID="deb"
set PASSWORD="deb"

%JAVA_HOME%\jre\bin\java -Djava.ext.dirs=%COMMONS%;%PLAIN_JAVA%;%JAVA_HOME%/jre/lib/ext -cp %COMMONS%;%PLAIN_JAVA% -jar automation.jar %REPOSITORY% %USERID% %PASSWORD% "deb"
pause

Summary

I hope this helps some people out there. I will try to add more details if I can find the time.

Posted in Jazz | Leave a comment

Build On State Change Work Item Save Participant


In the unlikely event you have missed it or just to complete the hit list if you search for examples on this blog, the Build On State Change Work Item Save participant/follow up action is a complete example as part of the Rational Team Concert Extensions Workshop.

The Build On State Change Work Item Save participant monitors work item state changes of configured work item types and state changes. If a qualifying change happens, it issues a build request for a configured build definition. The example comes with a complete package including the configuration UI.

Just Starting with API work?

If you just get started with extending Rational Team Concert, or create API based automation, start with the post Learning To Fly: Getting Started with the RTC Java API’s and follow the linked resources.

Posted in Jazz, RTC, RTC Extensibility, RTC Process Customization | Tagged , , , , | Leave a comment

RTC Process Customization – What you can and cannot do


Rational Team Concert Process Customization – What you can and cannot do, that is the title of the webinar I presented two days ago.

If you are interested in my view on this, you can find the replay of the webinar here in the Rational Team Concert Enlightenment Series.

The slides are shared here.

 

Posted in Jazz, RTC Automation, RTC Extensibility, RTC Process Customization | Tagged , , , | Leave a comment

Upgrading your CLM system? – Definitive checklist.


Upgrading the IBM CLM solution is sometimes received as complex and users struggle with it. Yesterday I was notified that there was a blog from a colleague providing a check list that helps making this easier and more likely successful. I thought this is worth reblogging to make it easier to find.

Please find the information in Paul’s blog post Upgrading your CLM system? – Definitive must-read checklist just published. The link points to our Deployment Wiki which is always worth looking at for all things related to deployment, install, setup and configuration.

Posted in CLM, Jazz, RTC, Upgrade | Tagged , , | Leave a comment

How to filter RTC email notifications by project or by user


Spamming the in-boxes of your project with notification mails? This is a reblog of Marks internal blog.

The RTC Mail Filtering Problem

A common requirement for Rational Team Concert administrators is the need for limiting email notifications for a single RTC project or user.  There are many scenarios which might drive this need.  Here are some examples:

  • Thousands of work items are imported and you want to not cause notifications for the new work items.
  • A new field is added and data from the old field is copied over, affecting many work items.  We don’t want users to be notified.
  • A subset of users now use a different RTC server, and they no longer want notifications from the old project, but they need to remain active in the old project.

Unfortunately, the current RTC product as of version 6.0.1 does not support this requirement.  Notifications are controlled at the Jazz Team Server level.  The JTS may control multiple RTC, Rational Quality Manager, and Rational Doors Next Generation repositories.  Turning off notifications in the JTS turns off mail for all RTC, RQM, and RDNG projects from all repositories controlled by that JTS server.  Mail  generated for all of those RTC, RQM, and RDNG projects while notifications are off in the JTS are lost.
This leaves administrators with a tough choice:  lose all mail for everything connected to the JTS, or live with excessive and unwanted notifications for a single project.

A Solution:  Milters

We have found and implemented a solution for this requirement, milters.  Milters are plugins for Sendmail that add additional functionality to Sendmail.  The “milter-regex” milter plugin permits filtering mail using regular expressions.

These are the overall steps for utilizing the regex milter:

  • Set up a machine with Sendmail and install the milter-regex plugin
  • Configure Sendmail to allow mail from the JTS server machine in /etc/mail/access
  • Develop a set of regular expressions which cause mail from a particular project or user to be found and filtered out and update the /etc/mail/milter-regex.conf configuration file
  • Restart the milter-regex service
  • Edit the email settings of the JTS to point to your Sendmail machine as the mail SMTP Server

You can easily add and remove projects and users from the filtering list by editing the /etc/mail/milter-regex.conf file.  You can turn off filtering completely by restoring the JTS SMTP Server value back to its original setting.

Regular Expressions

You’ll have to examine your email templates to determine the project and user information for the various email formats.  Here’s an example of the configuration statements for our email templates to filter out all mail for project “Project XYZ”:

reject “Mail filtered for project: Project XYZ”
body ,Team Area:.* Project XYZ,i
body ,Project Area:.* Project XYZ,i

Both “body” statements are required.  The “reject” statement defines a message which is logged into the /var/log/messages log file when a piece of mail is filtered out.

This is an example of filtering for a single user:

reject “Mail filtered for user: Mark E. Ingebretson”
body ,The user ‘Mark E. Ingebretson’ made a .* request,i*
body ,Mark E. Ingebretson mentioned you in,i
body ,Mark E. Ingebretson.*changed on,i

If you are a user of our IBM Systems servers and need email filtering, you can submit an ITHELP request.

Other Approaches

There is another approach from Sam provided on the Jazz.net forum here: Manage User E-mail Preferences for Mass Updates. It has been sitting on my Interesting Links page for a while. Time to show it here.

Summary

I found the topic very interesting and related questions also come up on Jazz.net, so I decided to re-blog and promote this when Mark showed this to me. I hope it helps our users all over the world. I hope that this solution can help other RTC administrators address this important requirement.

Posted in CLM, Jazz, RTC, RTC Automation, RTC Process Customization | Tagged , , | 4 Comments

JavaScript Attribute Customization – Where is the log file?


A lot of users try Java Script based attribute customization and often run into issues. They ask on the Jazz.net forum to get the issue solved. Unfortunately the questions usually lack the information required to help. This post explains how to retrieve log information to be able to provide this information.

Where are the Script Log files?

Java Script attribute customization can use the console to log text messages into a log file.

console.log("My message");

The question is, where are the log files?

The script context

Java Script attribute customization scripts are, as far as I can tell, run in one of the following contexts:

  1. The Eclipse Client
  2. The Web Browser
  3. The RTC Server

Dependent on the context it is run, the log information can be found in a log file that is created and maintained by the

  1. The Eclipse Client
  2. The RTC Server

Please note that the logging information is not in the RTC Application log file CCM.log.

The Jazz.net Wiki entry about attribute customization provides hints about how to log data and how to log data and how to debug scripts in the section Debugging Scripts. Similar information is provided in the Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management.

Unfortunately both only talk about how to find the server log information for Tomcat. Since Websphere Application Server and WAS Liberty are also valid options, how can one find the log files in this case?

Find The Eclipse Workspace Log

As background, note that the Eclipse Client as well as the RTC Server are based on Eclipse technology. This common technology is used to log the data and determines the log file location and name.

Each running Eclipse has a workspace location and stores meta data and log information in this workspace. The workspace is basically a folder in the file system. The metadata is stored in a sub folder with the name .metadata. The log file is in this folder and named .log.

For the RTC Eclipse client and for scripts that run in this context, open the Eclipse workspace folder that is used and find the .log file in the .metadata folder.

For the RTC Server, the easiest way to find this workspace and the enclosed log file that I have been able to find is to search for the folder .metadata. For Tomcat and WAS Liberty standard installs go to the folder where the RTC Server was installed and then into the sub folder server. From here search for the folder .metadata.

For Websphere Application Server (WAS) go the profile folder for the profile that includes the RTC server deploy and search there.

Here an example search for a test install based on WAS Liberty:

LogFileLocations_2016-06-17_13-10-14

Note that every Jazz application has its own Eclipse workspace with metadata folder and workspace log file. The one interesting for RTC attribute customization is the workspace of the RTC server. The folder structure includes the context root of the Jazz application. Each application has a different context root which typically matches the application war files name prefix. The RTC application typically has the context root and application war files name prefix ccm. Open the workspace for this application and find the log file.

Looking Into the Log File

You can look into the log file. Please make sure to use a tool that does not block writing to the log file, while you are browsing its content. The log file is kept open by the server when it is running and blocking it from writing is not what is desired. Use more or an editor that reads the file and does not block it. For windows users: notepad does lock the file for writing. Use a different tool such as notepad++.

The Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management provides some examples for how logs look like and can be created. If you can’t find the log entry you are looking for, always check the server log as well. Maybe the script runs in a different context than you expect.

Here an example for log entries:

Log_Examples_2016-06-17_14-58-43

Load Errors

It might happen, that an expected log entry is not found in any of the log files. In this case make sure to check for script loading errors as well as thrown exceptions at the time the script was supposed to run.

Load errors can be caused by different reasons.

One reason can be that attachment scripts are not enabled. There are enough indicators in the Attribute customization editor in Eclipse that a user should have spotted this these days.

Another reason can be that the script is syntactically not correct and can not be interpreted as a valid JavaScript. One reason for a script not being recognizable as a valid script that I have seen recently is an incorrect encoding. If an external editor is used to edit the script and the script is then loaded from the file, make sure that the script has a correct UTF-8 encoding. If in doubt change the encoding to UTF-8 and reload the script.

Why would the encoding be important? The encoding controls the format of the content. it is hard to determine the encoding from a file and it is often not checked. But expecting a specific encoding but loading a file that was encoded in a different one can cause to find unexpected content. This can can cause the JavaScript not being recognized as JavaScript and the load fails.

Debugging vs. Logging

Using the debugging techniques explained in the Wiki entry in the section Debugging Scripts and in the Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management should be the preferred option and is usually more effective.

Looking at the logs is still a valid option, especially to be able to log execution times and to find script loading issues and for scripts that are run in the background in the server context, such as conditions.

I found using the Chrome Browser and the built in Developer Tools to be most effective. The scripts can easily be found in the sources tab under the node (no domain). Make sure to enable the debug mode as explained here: Debugging Scripts.

JavaScript_Debug_Chrome_2016-06-17_14-24-43

Summary

This post explains how to find the log files that contain log information written by JavaScript attribute customization scripts. I hope that this helps users out there and makes their work a little bit easier.

Posted in Internet Of Things, Jazz, RTC, RTC Process Customization, WAS Liberty Profile | Tagged , , | Leave a comment

Raspberry Pi Unleashed – Setup the GrovePi+


This is the next post in my series around the Raspberry Pi and the Internet of Things.. To talk about Internet of Things our Raspberry Pi should actually do something and provide some data or service. In this post we will set up the Raspberry Pi with the GrovePi I/O board and sensors. The Raspberry Pi will also be referred to as RPi in the posts.

Related Posts

  1. Raspberry Pi Unleashed – Getting Started with the Internet of Things and the Raspberry Pi
  2. Raspberry Pi Unleashed – Setup the GrovePi+ – this post

Start All Over

I already had setup the GrovePi+ once on a NOOBS based Raspbian Jessie full desktop image for testing. This was just to understand the process a bit and get used to the RPi. To be able to blog about it, I wanted to do this process again. So I decided that I wanted to start with a Raspbian Jessie Lite, a minimal image based on Debian Jessie. Devices in the Internet Of Things are probably usually only set up with the essential software they need to function. So I was curious what I needed to do to get this up and running.

Initial Install

So I started all over again.

After power up, the Raspberry Pi started without any issues and I was able to login using the Keyboard and my monitor. There is only a shell available in the new image.

PIE_With_Peripherals

Install Without a Display and Keyboard

It should be possible to avoid having to use a display and peripherals on the RPI to set it up, only using the network connection. The best approach would be to use an image instead of the NOOBS distribution. It is necessary to connect to the RPi once it is up. This requires to be able to see the IP address. In a local environment, you might have access to the router to be able to determine and even set the RPi’s IP address. In other environments, follow the tips in this blog to find the IP address using special tools such as nmap or Zenmap.

Once you have the IP address, use a SSH shell connection (e.g. using Putty on Windows) to the RPi. This allows copy and paste as well as screenshots to create documentation.

Use a SSH Shell Connection

The big display is more useful as a secondary display to the laptop. I set up my network so that the RPi always gets the same IP address for Ethernet and WLAN. At this point Ethernet was working  which allowed to connect to the RPi with SSH and to login.

Connected_1

This shell is going to be used to setup the system and add all needed packaging.

Install GIT

From my prior experiments I knew that I would need GIT to be able to set up the GrovePi+. The Raspbian Jessie Lite does not have GIT installed – i tried running GIT from the shell, and the program was not found. A quick search on the internet shows that this can be changed easily by running

sudo apt-get install git-core

The image loaded all needed packages and installed GIT. I had to press ‘y’ to  perform the install process.

Install the GrovePi software

Shutdown and Switch Off

It is necessary to shutdown the system before switching it off. By running

sudo shutdown

The shutdown is initiated. The connection will be lost after some time and it is safe to switch power off.

Connect GrovePi+ Board

Make sure the Raspberry Pi is shutdown and power is off.  Unfold the first section Connect the GrovePi+ to the Raspberry Pi in the description to connect your GrovePi+ to the Raspberry Pi.

Setting Up The Software for the GrovePi+

Make sure everything is nicely connected and start the RPi again. Connect to the RPi and log in.

To run the GrovePi+ it is necessary to install some additional software. The manufacturer provides this Getting Started description for the steps. I found this a bit confusing, as some of the steps refer to a special Raspbian image I don’t have here. The following steps worked for me.

Follow this link to set up the software for the GrovePi+. The link can be found on the right side of the Getting Started description.

The original description uses sub folders in the folder /home/pi/Desktop to install this software. Since there is no UI and thus no desktop the following steps install the necessary software into sub folders of the folder /home/pi/.

The first step is to download the software and drivers using GIT.

cd /home/pi/
sudo git clone https://github.com/DexterInd/GrovePi.git

It will take a short while to download the software. Once the data is available locally in the folder GrovePi, it can be installed by running

cd /home/pi/GrovePi/Script
sudo chmod +x install.sh
sudo ./install.sh

Please note, all the executable and shell scripts in the folder tree cloned by GIT are missing the executable permission, so running a script always requires to set the executable permission, this is a common pattern.

After the install was performed the system will automatically reboot, if this is not prevented. In any case, if you haven’t already connected the  GrovePi+ board, shut down the Raspberry Pi, switch off the power and install the GrovePi+ board.

To test if the GrovePi+ board is available you can run

sudo i2cdetect -y 1

The result should look like the image below:

Test_GrovePi

If you can see a “04” in the output, this means the Raspberry Pi is able to detect the GrovePi+ board. What it does is to check for I2C ports. I2C is a connection standard that is often used as interface for sensors and devices. You can find the technical details and tutorial for the GrovePi here. This includes the information about the available connection ports here.

If you followed my advice to go for option 2 in the last post and bought a LED for the GrovePi+,   you can test that now as described in step 10 in this link to set up the software for the GrovePi.

If you followed Tim’s shopping list you can use the Grove Barometer for testing.

If you haven’t connected your sensor or device to your GrovePi+, you should shutdown and power down the RPi. It is always a good idea to power off if connecting something to a hardware, if not stated otherwise.

Look up the connection method for your sensor or device. Connect the device to the port required.

Example The Grove Barometer sensor details for the RPi are described here in the Wiki. Look at the GrovePi+ port description and look up an I2C port. Connect the Grove Barometer Sensor to the chosen port.

Power up the RPi, open a connection and log in.

Example the Grove Barometer Sensor. Make the sensor Python scripts executable.

# Make the high accuracy barometer sensor example script executable
cd /home/pi/GrovePi/Software/Python/grove_barometer_sensors/high_accuracy_hp206c_barometer 
sudo chmod +x high_accuracy_barometer_example.py

Once this has been performed successfully, run the sensor example script to read the sensor like below

# Read high accuracy barometer sensor using the example script
cd /home/pi/GrovePi/Software/Python/grove_barometer_sensors/high_accuracy_hp206c_barometer 
sudo ./high_accuracy_barometer_example.py

You should see something similar to the image below.

Barometer_Sensor_Test

The sensor was successfully read and you can now go ahead and use the GrovePi+ board and the I/O devices you purchased for it. If you run into issues, you might wnat to upgrade the firmware for the GrovePi+ if you have not already done so.

Firmware Update for the GrovePi+

It is always a good idea to make sure your peripherals have the latest firmware. The procedure to upgrade the firmware for the Grove PI+ is described here.

Essentially you perform the steps below

# Firmware update 
cd /home/pi/GrovePi/Firmware
sudo chmod +x firmware_update.sh
sudo ./firmware_update.sh

I ended up having problems with the first NOOBS Raspbian image I used and the described procedure was unable to locate the GrovePi+ board. The sensor board was not recognized and the update never started. A blog entry hinted to use the Raspbian_For_Robots update scrips instead which worked.

# Firmware update Raspbian_For_Robots
cd /home/pi/
sudo git clone https://github.com/DexterInd/Raspbian_For_Robots.git

cd /home/pi/Raspbian_For_Robots/upd_script
sudo chmod +x update_GrovePi.sh
sudo ./update_GrovePi.sh

sudo chmod +x update_GrovePi_Firmware.sh
sudo ./update_GrovePi_Firmware.sh

Enable the Wireless Connection

After getting rid of the dedicated monitor, keyboard and mouse for the Raspberry Pi it was time to see how to enable the wireless connection with this image. A search in the internet provides with this Wiki page how to set up WIFI in Raspbian using the command line.

The steps are simple, assuming there is a Edimax Wi-Fi USB Adapter Nano Size or another supported WIFI USB adapter connected to the RPi. Start a detection run to find the available networks like this.

# Detect networks
sudo iwlist wlan0 scan

You should get a list of networks with details such as the ESSID and the authentication used, similar to below:

Wireless

As described in p WIFI in Raspbian using the command line edit the configuration file and add the required information.

# Edit WIFI configuration file
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Modify the file as described. My result was like beow. I changed the country code and added the network that should be chosen.

Wireless_Config

After saving the configuration changes and rebooting (the other suggested measures did not work for me) my Raspberry Pi was connected to the wireless network.

Please note, Like for the Ethernet connection I configured my VDSL router/wireless network to provide the same IP address to the Raspberry Pi. This allows to use a remote connection reliably in my network. Otherwise I would have needed some DSL or other name resolution to be able to reliably find the IP address for my Raspberry Pi to open the wireless SSH connection.

Up and Running

All the important parts are now up and running on the device and most of the periphery that made it look like a desktop commuter is gone. Now it is ready to join the Internet of Things.

PiWireless

Summary

Now the Raspberry Pi is configured to use the GrovePi+ I/O board and the sensors. The driver software is available.

This was actually really easy to do and the documentation I found was really good enough to get me going really quickly. This is no comparison to bringing up a custom RTOS on a custom board where you might even have to create your own device drivers to get the system working. I am not sure if python scripts can be debugged. I am not yet sure if it is possible to set up a cross development and debugging environment on my laptop to develop and deploy on the RPi, that remains to be seen.

I will now have a look at how to connect to Bluemix to provide some data/service and try to get my thing into the internet…… Hmmmm, that does not sound right.

Posted in Internet Of Things | Tagged , | Leave a comment