EWM Extensions Workshop remote debugging in 7.0.x

As explained in Issues with the EWM/RTC Extensions workshop in 7.0.x versions, I experienced issues with configuring the EWM Development server for remote debugging. The first EWM version I experienced this, is 7.0.1.

As mitigation I suggested to replace the JRE with an earlier one, or one that works. Unfortunately, this does not look sustainable. I have tried downloading the newest IBM Java SDK and the issue still happens.

I have looked a little bit deeper into it and discovered that the server.startup script provides a debug option. Using this option allows remote debugging the EWM server. I am looking into an update to the RTC/EWM Extensions Workshop for 7.0.x. until this is available here a procedure how to follow the workshop for 7.0.x versions.

Update for the Extensions Workshop now available

All the issues have been addressed in an update to the RTC Extensions Workshop for the 7.0.x and later versions. This should address all the issues.

Lab 1.2 – Do not add debug arguments.

In Lab 1.2, do not add the following lines to the server.startup script, remove these lines:

set JAVA_OPTS=%JAVA_OPTS% -Xdebug
set JAVA_OPTS=%JAVA_OPTS% -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=3388

Scroll to the bottom of the server.startup script and inspect the statements behind setting the JAVA_OPTS. Note that there are an action and an option supported.

Special parameters supported for starting Liberty Profile

Start server for remote debugging

The action -debug can be used to start the Liberty Profile Server in debug mode. When following the workshop, continue to Lab 1.5.

Follow the lab until 1.5__32 Add the following steps:

Open a console/cmd window and change directory into the server folder containing the server.startup script. Stop the development server by running server.shutdown.

>server.shutdown

Wait until the server is successfully stopped. Start the development server by running the following command.

>server.startup -debug

Note that the server does not start, it suspends listening for a remote connection on a dt_socket/port. The default socket/port address is 8000.

Server waits in debug console

The server will not continue starting unless a remote connection is opened on that port.

Connect Eclipse as remote debugger

Follow the workshop and open the [RTCExt] Debug Running Liberty Profile launch configuration. This launch is used to connect to a remote Java application. In our case the application is the RTS/EWM development server waiting on socket/port 8000.

Change the port in the launch to 8000 then click Apply and Debug.

Change debug port to 8000 apply and start debugging.

Switch to the command prompt. It takes a while and the server continues to start. Wait for the server application to successfully start.

Continue with the workshop and remote debug the server.

When disconnecting from the remote Java application, the RTC development server will show again, that it waits for a remote connection.

Reprovision the Jazz applications

The option -clean can be used to force to re-provision the Jazz Applications installed on the Liberty Profile server. This is necessary to force rereading custom server extensions during deployment of these extensions. The option can also used together with the -debug action.

Summary

Following the solution described above, allows to successfully perform the Lab 1.5 of the Rational Team Concert Extensions Workshop. I am currently trying to find out if there

  • Is an option to change the default port for this option
  • Is an option to avoid suspending the server startup until the remote debugger connection is initially done

I will update the workshop and publish the changes. I have already done several other changes to adjust the workshop for the newer EWM/RTC versions.

As always, I hope that the blog posts here help users in the Jazz community.

JavaScript Attribute Customization – Where is the log file?

A lot of users try Java Script based attribute customization and often run into issues. They ask on the Jazz.net forum to get the issue solved. Unfortunately the questions usually lack the information required to help. This post explains how to retrieve log information to be able to provide this information.

Where are the Script Log files?

Java Script attribute customization can use the console to log text messages into a log file.

console.log("My message");

The question is, where are the log files?

The script context

Java Script attribute customization scripts are, as far as I can tell, run in one of the following contexts:

  1. The Eclipse Client
  2. The Web Browser
  3. The RTC Server

Dependent on the context it is run, the log information can be found in a log file that is created and maintained by the

  1. The Eclipse Client
  2. The RTC Server

Please note that the logging information is not in the RTC Application log file CCM.log.

The Jazz.net Wiki entry about attribute customization provides hints about how to log data and how to log data and how to debug scripts in the section Debugging Scripts. Similar information is provided in the Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management.

Unfortunately both only talk about how to find the server log information for Tomcat. Since Websphere Application Server and WAS Liberty are also valid options, how can one find the log files in this case?

Find The Eclipse Workspace Log

As background, note that the Eclipse Client as well as the RTC Server are based on Eclipse technology. This common technology is used to log the data and determines the log file location and name.

Each running Eclipse has a workspace location and stores meta data and log information in this workspace. The workspace is basically a folder in the file system. The metadata is stored in a sub folder with the name .metadata. The log file is in this folder and named .log.

For the RTC Eclipse client and for scripts that run in this context, open the Eclipse workspace folder that is used and find the .log file in the .metadata folder.

For the RTC Server, the easiest way to find this workspace and the enclosed log file that I have been able to find is to search for the folder .metadata. For Tomcat and WAS Liberty standard installs go to the folder where the RTC Server was installed and then into the sub folder server. From here search for the folder .metadata.

For Websphere Application Server (WAS) go the profile folder for the profile that includes the RTC server deploy and search there.

Here an example search for a test install based on WAS Liberty:

LogFileLocations_2016-06-17_13-10-14

Note that every Jazz application has its own Eclipse workspace with metadata folder and workspace log file. The one interesting for RTC attribute customization is the workspace of the RTC server. The folder structure includes the context root of the Jazz application. Each application has a different context root which typically matches the application war files name prefix. The RTC application typically has the context root and application war files name prefix ccm. Open the workspace for this application and find the log file.

Looking Into the Log File

You can look into the log file. Please make sure to use a tool that does not block writing to the log file, while you are browsing its content. The log file is kept open by the server when it is running and blocking it from writing is not what is desired. Use more or an editor that reads the file and does not block it. For windows users: notepad does lock the file for writing. Use a different tool such as notepad++.

The Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management provides some examples for how logs look like and can be created. If you can’t find the log entry you are looking for, always check the server log as well. Maybe the script runs in a different context than you expect.

Here an example for log entries:

Log_Examples_2016-06-17_14-58-43

Load Errors

It might happen, that an expected log entry is not found in any of the log files. In this case make sure to check for script loading errors as well as thrown exceptions at the time the script was supposed to run.

Load errors can be caused by different reasons.

One reason can be that attachment scripts are not enabled. There are enough indicators in the Attribute customization editor in Eclipse that a user should have spotted this these days.

Another reason can be that the script is syntactically not correct and can not be interpreted as a valid JavaScript. One reason for a script not being recognizable as a valid script that I have seen recently is an incorrect encoding. If an external editor is used to edit the script and the script is then loaded from the file, make sure that the script has a correct UTF-8 encoding. If in doubt change the encoding to UTF-8 and reload the script.

Why would the encoding be important? The encoding controls the format of the content. it is hard to determine the encoding from a file and it is often not checked. But expecting a specific encoding but loading a file that was encoded in a different one can cause to find unexpected content. This can can cause the JavaScript not being recognized as JavaScript and the load fails.

Debugging vs. Logging

Using the debugging techniques explained in the Wiki entry in the section Debugging Scripts and in the Process Enactment Workshop for the Rational solution for Collaborative Lifecycle Management should be the preferred option and is usually more effective.

Looking at the logs is still a valid option, especially to be able to log execution times and to find script loading issues and for scripts that are run in the background in the server context, such as conditions.

I found using the Chrome Browser and the built in Developer Tools to be most effective. The scripts can easily be found in the sources tab under the node (no domain). Make sure to enable the debug mode as explained here: Debugging Scripts.

JavaScript_Debug_Chrome_2016-06-17_14-24-43

Summary

This post explains how to find the log files that contain log information written by JavaScript attribute customization scripts. I hope that this helps users out there and makes their work a little bit easier.

Publishing XML and Other Data With the WAS Liberty Profile

Since CLM now ships the WebSphere Application Server Liberty Profile as default, you might want to be able to publish XML and other data similar to the easy way that was available in Tomcat, by just using the application server that is available. This avoids having to set up an additional Web or application server.

Problem

Sometimes it is necessary to provide access to some files using the HTTP/HTTPS protocol.

This is interesting for the Process Enactment Workshop, or if you want to publish your own XML data to be used with HTTP Filtered Value Sets, host custom Doors Next Extensions, or if you want to publish Build results.

It is convenient if there is already an application server available to host these files there and make them accessible, instead of having to set up a web server. This especially holds if it is necessary to provide the data using HTTPS, and not HTTP, because browsers these days reject to process content that is mixed from HTTP and HTTPS sources. The latter is important for Doors Next Generation extensions. Not having to set up a new server also avoids having to create a trusted certificate for it. It reduces the additional failure point, backup and maintenance and provides URI stability as well, since you don’t want to change the URI for the CLM applications hosted anyway.

Tomcat

With Tomcat, it was relatively easy to do that, by just adding a folder under \tomcat\webapps and placing your files there. Assuming the folder name is myfiles, Tomcat then provides the files with the public URI root https://serverpath:port/myfiles/fileName.

Another way was to simply drop the file into the tomcat/webapps/ROOT/ or a sub folder, which has the same effect.

Is there a similar way to achieve this with the WAS Liberty Profile shipped with CLM?

Example Scenario

Lets assume, like in the scenario explained in Publish and Host XML Data Using Tomcat – The Easy Way, we want to publish a file makers.xml to be accessible with a context root PEWEnactmentData on the server with the public URI root https://clm.example.com:9443/.

The expected outcome is to be able to access the XML file content using the URL https://clm.example.com:9443/PEWEnactmentData/makers.xml.

Solutions

There are two relative simple solutions for WAS Liberty. The solutions below are based on information from Lars, one of our WebSphere experts and the IBM WebSphere Application Server Liberty Profile Guide for Developers.

For WAS Liberty Profile, all solutions require to deploy an application. There are several ways to create and deploy an application in WAS Liberty Profile.

  1. Deploy an application in some folder and declare the context root and location of the application in the server.xml or specific location XML file
  2. Deploy an application in the dropins folder; for this to work, it is necessary to make sure the application server has the dropins monitoring enabled for this to work

Deploying an application in this context means

  1. Create a folder with some contend in a location (or copy the folder into the location)
  2. Deploy a compressed folder with some content

Please note: Any changes to any of the server configuration XML files are automatically picked up be WAS Liberty Profile. There is no need to reboot the server for the steps below to work.

1. Deploying an Application – Standard

The trick in this solution, is to pretend to deploy an uncompressed Web Application such as a WAR file. It is obviously possible to create a real WAR file or the structure that is contained in it, but that is not really necessary. See the last section here if you are interested in deploying the application as compressed WAR file.

1.1 Deploying the Application

The only thing necessary to trick the WAS Liberty Profile into making the data available with a correct context root is to create a folder with the context root as name and an extension .war, and to make the application location known in an XML file.

In the default setup of WAS Liberty Profile with CLM, after the first launch, all installed CLM applications end up in the folder <Install Folder>\server\liberty\servers\clm\apps. The screen shot below shows this structure.

FolderStructure

The applications are deployed as compressed WAR file and have been extracted to a folder with the name of the application. As an example the CCM application (RTC) came shipped as ccm.war.zip and was extracted to a folder ccm.war. The context root of the CCM application is ccm, which is basically the name without the extension war. The same pattern holds for all the other applications.

It is possible to create a folder to host the files to be published. The folder should be named <context root>.war to result in an application with the desired context root.

Files within this folder will be available with their file name.

A file in that folder with the name <filename> will be accessible as <public URI root>/<context root>/<filename>.

If the files are within sub folders, the folder path within the root folder and the file name compose the file name part of the URL. For example, the root folder has a sub folder names myfiles and a file test.xml in it, the file is accessible as <public URI root>/<context root>/myfiles/test.xml.

In our example with the given public URI root being https://clm.example.com:9443/, the folder name <context root> being PEWEnactmentData and <filename> being makers.xml the URL would be https://clm.example.com:9443/PEWEnactmentData/makers.xml.

1.1 Declaring the Application Location

The WAS Liberty Profile needs to know which application has to be published and where to find its resources. WAS Liberty Profile defines an XML element to declare which application is located where. The element is <appliction…./>. It supports various attributes and can come in many different forms. The most important attributes are

  • id: Must be unique and is used internally by the serve
  • location: The path to the application resource
  • context-root: The context root of the application
  • name: The name of the application
  • type: Specifies the type of application (war or ear)

The location and ID is required and either the name equal to the context root, or the context-root can be provided. The type can be omitted. I did not provide the ID and it worked for me, but since the server requires it, it should be provided. Use the same value as the name or context root for the ID.

It is necessary to add the application declaration to the server configuration for example by adding it to the server.xml file. In a CLM deployment a file appliction.xml is included in the server.xml, declaring all the applications and it makes sense to use that file instead.

Save the xml files after any change to activate the new setting. The server will pick up the change.

2. Deploying an Application – dropins

The WAS Liberty Profile has the capability to monitor a special dropins folder on a regular basis. If new content is detected, it is automatically made available by the server. The automatic monitoring needs to be enabled for this to work. A standard CLM deployment disables the monitoring to reduce server load.

2.1 Deploying the Application – dropins

In a default CLM install the dropins folder is located here:

<Install Folder>\server\liberty\servers\clm\dropins

Create a folder named <context root>.war in the dropins folder and add the files to publish. After detecting the addition, the files will be accessible like explained in 1.1.

The folder <Install Folder>\server\liberty\servers\clm\dropins should now look like below.

Published application with file

2.1 Enable Automatic Monitoring for Dropins

The automatic monitoring of the dropins folder is disabled for a standard CLM setup with WAS Liberty Profile. This saves processor cycles and I/O for the server operation. It has to be enabled once in the server.xml file for this solution to work. Find the XML element <applictionMonitor> and change the attribute dropinsEnabled from “false” to “true” and save the file. The image below shows the changed configuration file.

Enable Dropins Monitoring

The attribute pollingRate determines how often the server checks the location for changes. Adjust the value to your needs.

Save the server.xml file after any change to activate the new setting. The server will pick up the change.

The XML file content should now be accessible.

Example 1 – Standard

In the example we want to publish a document with the URI https://clm.example.com:9443/<context root>/makers.xml, where <context root>=PEWEnactmentData. The public URI root is already given by the server set up.

1.1 Create the “Application”

As first step create the application. Create a new folder named PEWEnactmentData.war in the folder <Install Folder>\server\liberty\servers\clm\apps. The folder could be created anywhere, but it makes sense to put it where the other applications are. If it is necessary to use a different folder, for example to allow other users to modify the content, place the folder in a different location and note the path. The path to the location will become important in the next step.

Put the files to be published underneath this folder. In the given example, put the file makers.xml into the folder.

The folder structure should look like the image below:

New Application folder

1.2 Publish the “Application”

The WAS Liberty Profile needs to know which application has to be published and where to find its resources. WAS Liberty Profile defines an XML element to declare which application is located where. The element is <appliction…./>. It supports various attributes and can come in many different forms. The most important attributes are

  • id: Must be unique and is used internally by the serve
  • location: The path to the application resource
  • context-root: The context root of the application
  • name: The name of the application
  • type: Specifies the type of application (war or ear)

The location and ID is required and either the name equal to the context root, or the context-root can be provided. The type can be omitted. I did not provide the ID and it worked for me, but since the server requires it, it should be provided. Use the same value as the name or context root for the ID.

The element can be put into several places. In a CLM installation, by default, the applications are declared in the file

<Install Folder>\server\liberty\servers\clm\conf\application.xml

The file is included in the file <Install Folder>\server\liberty\servers\clm\server.xml. It would be possible to place the application declaration in the file server.xml, but it seems to make more sense to put it into the file application.xml instead.

Add lines

<!–  My Application –>

<application id=”PEWEnactmentData” name=”PEWEnactmentData” location=”${server.config.dir}/apps/PEWEnactmentData.war”/>

to the file application.xml and save the file.

The file should look like below:

Declare new Application

In this case the location is relative to the server configuration folder which n our case is <Install Folder>\server\liberty\servers\clm\. The location could really be anywhere. If it is necessary to include a folder that, as an example, can be accessed by regular users, you could change the location to somewhere with less read and modification restrictions.

After the save of the application.xml, the published XML file should be accessible using the URL https://clm.example.com:9443/PEWEnactmentData/makers.xml.

The image below shows the browser access.

Browser Access To XML File

2. Deploying an Application – dropins

In the example we want to publish a document with the URI https://clm.example.com:9443/<context root>/makers.xml, where <context root>=PEWEnactmentData. The public URI root is already given by the server set up.

2.1 Create the “Application”

As first step create the application. Create a new folder named PEWEnactmentData.war in the folder <Install Folder>\server\liberty\servers\clm\dropins.

Put the files to be published underneath this folder. In the given example, put the file makers.xml into the folder.

The folder structure should look like the image below.

Published application with file

Assuming the automatic application monitoring is enabled, the file should be accessible after the next polling cycle, (at least after waiting the full pollingRate). If automatic application monitoring is not enabled, enable it as described in 2.2 and save the configuration file.

The XML file content should now be accessible using the URL https://clm.example.com:9443/PEWEnactmentData/makers.xml.

Deploying Compressed Folder Structures

It is also possible to publish real WAR, EAR similar to the way described above. For example create a WAR file as described in Publish XML Data Using Tomcat – Hotfix for The Process Enactment Workshop and attached to that post. Deploy the compressed WAR file, for example PEWEnactmentData.war in the dropins folder or declare it as an application and put it onto another location like explained below. This automatically deploys the application and makes the contained files accessible.

Just zipping the folder structure we created above into a zip file, name it PEWEnactmentData.war and trying to deploy does not seem to be working however. The WAR file needs to provide required supporting structures in this case.

Related Posts

The RM Extensions Hosting Guide for CLM 6.0.1 and later versions explain the dropins example as well.

Summary

This post shows how easy it is to publish supporting files on the WAS Liberty Profile. This is important if you want to host custom Doors Next Extensions, or if you want to publish your own XML data to be used with HTTP Filtered Value Sets, for the Process Enactment Workshop and it can possibly also used with Build result publishing. In the latter case, you would likely publish a root folder that contains all the build results.

As always I hope this post has some value to users out there, or is at least interesting to read.