Adapters - Integrations Using TeamForge EventQ

EventQ Adapters

TeamForge EventQ collects metadata about key lifecycle activities from external sources like work item trackers, SCM repositories, CI/build servers, and code review systems. To do so, TeamForge EventQ relies on adapters to collect the metadata from sources. TeamForge EventQ packages the following adapters:

Source Association Keys

TeamForge EventQ uses source association keys to uniquely identify incoming data from activity sources that it monitors.

Adapters supply data to TeamForge EventQ through the message queue (MQ) layer. When a relevant activity occurs in source systems, TeamForge EventQ adapters generate a message with key metadata about the activity and place that message on the designated MQ server. When TeamForge EventQ receives the message, it must associate it to the proper source somehow. To do this, TeamForge EventQ supplies a unique “source association key” for each external source created. Adapters must supply this unique source association key with each message for the data to correlate with the appropriate project source.

Note that commit sources based on TeamForge project repositories do not need source associations keys since they are internal and are identified by other means. All other sources require source association keys.

Locate a Source Association Key

Source association key for existing activity sources are located on the corresponding source configuration page.

1. From the activity stream, click Manage Sources.
2. Select the desired source type toggle. TeamForge EventQ displays all existing sources.
3. Click Edit on the source whose key you wish to obtain. You see the configuration page for the selected source.
4. Locate the source association key for the selected source. On most browsers you can copy the key by clicking the small clipboard icon.

Jenkins Integration

The TeamForge/EventQ Jenkins adapter supplies build data from your Jenkins continuous integration server to TeamForge/EventQ for purposes of metadata archival, traceability and event reporting.

The TeamForge/EventQ Jenkins integration brings associations and traceability to Jenkins. The adapter displays associations between Jenkins builds and other tools within the TeamForge landscape like version control commits and binary artifacts. Traceability is therefore possible between Jenkins build, TeamForge, and various orchestrated third-party tools.

Inside TeamForge, Jenkins build details are visible in activity streams and association visualizations. Within Jenkins, associations are visible from the TeamForge Associations tab.

Clicking Trace provides a view of the build in the context of other associated activities. Use cases include root cause analysis and auditable traceability.

The TeamForge/EventQ Jenkins adapter is packaged as a Jenkins plugin. Once installed and configured, the plugin supplies build results metadata for configured Jenkins jobs to TeamForge/EventQ.

Jenkins Version Information

The following table lists the compatible Jenkins and TeamForge-Jenkins Plugin version.

Install a Jenkins Adapter

A new version of Jenkins integration plugin, v2.0.6, is released with TeamForge 18.3, which if configured can notify either the native TeamForge Webhooks-based Event Broker or EventQ about the build data.

CollabNet Plugin Features

• Notify EventQ when builds complete. The CollabNet Plugin must be installed once on each Jenkins server you wish to connect to TeamForge/EventQ.
• Notify TeamForge Webhooks-based Event Broker when builds complete.
• Authenticate users from TeamForge. If setup as the “Build & Test” application, it can even use Single Sign-On.
• Authorization from TeamForge, including the ability to set permissions in Jenkins based on roles in your TeamForge project.
• Upload the build log or workspace artifacts to the TeamForge Documents.
• Upload workspace artifacts to the TeamForge File Release System, as a post-build publishing task or as a build promotion task.
• Open/update/close TeamForge Tracker artifacts based on the Jenkins build status.

• Upload workspace artifacts to the Lab Management Project Build Library. (Requires CollabNet Lab Management).

  Click here to know more about the requirements for installing the latest CollabNet plugin.

Use the following instructions to have the Jenkins integration plugin notify the EventQ when a build is completed.

1. If you are integrating TeamForge and Jenkins for the first time:
   1. Log on to the Jenkins Server as a previliged Jenkins user, navigate to Manage Jenkins > Manage Plugins > Available.
   2. Select the latest CollabNet Plugin and install the plugin.
   3. Restart your Jenkins server.
2. Existing TeamForge-Jenkins integrations that use CollabNet Plugin v2.0.4 (or earlier):
   1. Log on to the Jenkins Server as a privileged Jenkins user, navigate to Manage Jenkins > Manage Plugins > Updates.
   2. Select the latest CollabNet Plugin and install the plugin.
3. Existing TeamForge-Jenkins integrations that use EventQ Jenkins Adapter v2.0 (or earlier) plugin:
   1. Log on to the Jenkins Server as a privileged Jenkins user, navigate to Manage Jenkins > Manage Plugins > Installed.
   2. Select the EventQ Jenkins Adapter v2.0 and click Uninstall.
   3. Select the Available tab.
   4. Select the latest CollabNet Plugin and install the plugin.
   5. Download the migrate_jenkins_plugin.sh script and save it to <JENKINS_HOME_DIRECTORY>/jobs/.
      Tip: Jenkins default home directory is /var/lib/jenkins/.
   6. Change ownership of the migrate_jenkins_plugin.sh file.

      chmod 755 migrate_jenkins_plugin.sh
      

   7. Run the migrate_jenkins_plugin.sh script.

      ./migrate_jenkins_plugin.sh
      

Configure a Jenkins Adapter to Notify EventQ

Configure the Jenkins adapter to notify EventQ when builds complete.

Before you start the configuration, make sure that you have installed the Jenkins plugin using the web interface. The Jenkins adapter has a system configuration as well as job level configurations. The system configuration sets defaults to be used by subsequently configured job level configurations.

Configuring an Individual Jenkins Job

1. Create a build source and have the configuration details handy for the following steps.
2. As a privileged Jenkins user, locate the job you wish to report build data to TeamForge/EventQ and navigate to its configuration page.
3. Add a post-build action to Notify TeamForge/EventQ when a build completes.

   

4. Select the Notify EventQ check box.

   Warning: DO NOT select both the Notify TeamForge and Notify EventQ options together as the Jenkins build will become unstable and the notifier will show a warning message on the Jenkins console and the traceability view.
   1. Using the values you obtained in Step 1, copy and paste the Queue Server value into the MQ URI (server URL) field.
      Note: Include the amqp protocol and the port information.

   2. Copy and paste the queue username and password values obtained from Step 1 in the Server Username and Server Password fields.

   3. Populate the Build source association key field with the value obtained from Step 1. For more information about association keys, see Source Association Keys.

      

5. By default, the Optional TeamForge Association View check box is selected. If required, you can override the global configuration by entering the TeamForge URL and user credentials.

   

6. Save the job configuration.

7. Run a build to test the new configuration and verify configuration. Information and errors will be reported to your Jenkins log and to the build console.

Configuring the Default Jenkins Global Settings

Now that you’ve set up your first job level configuration, you can save some effort next time around by defining system values for the Server URL, Server Username and Server Password. Note that each job needs a unique Source Association Key, so that will still need to be configured per job.

1. Refer to Step 1 in Configuring an Individual Jenkins Job and obtain values for the CollabNet TeamForge URL, Username and Password.
2. As a privileged Jenkins user, navigate to Manage Jenkins > Configure System page on the target Jenkins server.
3. On the Jenkins configuration page, go to the Global CollabNet Configuration section.
4. Select the Allow Global CollabNet Configuration check box.
   1. Copy and paste the Queue Server value in the CollabNet TeamForge URL field in Jenkins.
      Important: Include the amqp protocol and the port information.
   2. Enter the Username and Password.
5. Save the configuration.

External Git Repository Integration

With a customized post-receive extension (hook) installed, Git servers external to TeamForge can notify TeamForge EventQ of Git “push” activity.

TeamForge project Git repositories are already wired to supply push activity data to TeamForge EventQ; no special configuration needed. This section instead details integration of Git repositories external to TeamForge.

The TeamForge EventQ Git adapter is packaged as a Git post-receive hook script. Once installed and configured, the hook script supplies push metadata for the selected repository to TeamForge EventQ. The push data is associated to the proper source using source association keys (see Source Association Keys). Configuration therefore requires that the message queue server URL and credentials be supplied, along with the appropriate source association key.

Install a Git Adapter

Install a Git adapter as a post-receive hook on Mac OS X/Linux.

Minimum Requirements:

• Git 1.8.3.1 or later.
• Python 2.6. Python v3.0 and later are not compatible.

Each external Git repository you wish to monitor with TeamForge EventQ requires the installation of a customized post-receive hook script as follows.

Install Git Post-receive Hooks

Contact the CollabNet Support to download the eventQ-git-adapter-[version].zip adapter file and unpack it inside your repository’s hooks directory.

For example, if your repository’s hooks directory is located at /path/to/repo.git/hooks, cd /path/to/repo.git/hooks, download the zip file to the current location, then unzip eventQ-git-adapter-1.0.zip.

This will unpack the following into your hooks directory:

• Files: LICENSE.txt, README.txt, post_receive_orc_publish_amqp.conf.sample, post_receive_orc_publish_amqp.py
• Directories: eventq/ and pika/

Configuration

1. In a web browser, log into TeamForge EventQ as a privileged EventQ user and edit (or add) the relevant Commit source you wish to associate with this Git repository. Manage Sources > Toggle Commit > Add or Edit a Source > External repository
2. On the Edit Source page, locate the section titled “Adapter Configuration Information” and copy all values; these values are used later in the configuration process.
3. Switch back to your command line console and copy post_receive_orc_publish_amqp.conf.sample to post_receive_orc_publish_amqp.conf.
4. Edit post_receive_orc_publish_amqp.conf and modify options for your specific installation, for example:
   • Logging Section
     • filename — Full path to the desired log file, for example, /tmp/post-commit.log.
     • level — The logging level, which is one of these: debug , info, warning, error, or critical.
   • RabbitMQ (amqp) Section
     • host — The hostname of the amqp server. Copy and paste the hostname from the Queue Server value from the Edit Source page in TeamForge EventQ.
     • port — The port number of the amqp server (default is 5672). Copy and paste the port number the Queue Server value from the Edit Source page in TeamForge EventQ.
     • username — Username used to access the amqp server. Copy and paste the queue username value from the Edit Source page in TeamForge EventQ.
     • password — Password used to access the amqp server. Copy and paste the queue password value from the Edit Source page in TeamForge EventQ.
   • TeamForge EventQ Section
     • source_association_key — key provided by TeamForge EventQ. Copy and paste the source association key value from the Edit Source page in TeamForge EventQ. For more on association kys, see Source Association Keys.

Activate Post-receive Hook

Copy or create a symlink called “post-receive” linking to post_receive_orc_publish_amqp.py.

External Subversion Repository Integration

With a customized post-commit extension (hook) installed, Subversion servers external to TeamForge can notify TeamForge EventQ of Subversion commit activity.

TeamForge project Subversion repositories are already wired to supply commit activity data to TeamForge EventQ; no special configuration needed. This section instead details integration of Subversion repositories external to TeamForge.

The TeamForge EventQ Subversion adapter is packaged as a Subversion post-commit hook script. Once installed and configured, the hook script supplies commit metadata for the selected repository to TeamForge EventQ. The commit data is associated to the proper source using source association keys (see Source Association Keys). Configuration therefore requires that the message queue server URL and credentials be supplied, along with the appropriate source association key.

Install a Subversion Adapter on Mac OS X/Linux

Install a Subversion adapter as a post-commit hook on Mac OS X/Linux.

Minimum Requirements:

• Subversion 1.7 or later.
• Python 2.6. Python v3.0 and later are not compatible.

Each external Subversion repository you wish to monitor with TeamForge EventQ requires the installation of a customized post-commit hook script as follows.

Install Subversion Post-commit Hooks

Contact the CollabNet Support to download the eventQ-subversion-adapter-[version].zip adapter file and unpack it inside your repository’s hooks directory.

For example, if your repository’s hooks directory is located at /path/to/svn/repos/my_repo/hooks, cd /path/to/svn/repos/my_repo/hooks, download the zip file to the current location, then unzip eventQ-svn-adapter-1.0.zip.

This will unpack the following into your hooks directory:

• LICENSE.txt
• README.txt
• configdefaults.py
• configwrapper.py
• eventq_commit_gate_hook.py
• orchestrator_commit_gate_hook_snippet.sh
• pika/
• post-commit.bat.sample
• post_commit_orc_publish_amqp.conf.sample
• post_commit_orc_publish_amqp.py

Configuration

1. In a web browser, log into TeamForge EventQ as a privileged EventQ user and edit (or add) the relevant Commit source you wish to associate with this Subversion repository. Manage Sources > Toggle Commit > Add or Edit a Source > External repository
2. On the Edit Source page, locate the section titled “Adapter Configuration Information” and copy all values; these values are used later in the configuration process.
3. Switch back to your command line console and copy post_commit_orc_publish_amqp.conf.sample to post_commit_orc_publish_amqp.conf.
4. Edit post_commit_orc_publish_amqp.conf and modify options for your specific installation, for example:
   • Logging Section
     • filename — Full path to the desired log file, for example, /tmp/post-commit.log.
     • level — The logging level, which is one of these: debug , info, warning, error, or critical.
   • svnlook Section
     • svnlook — The path to your svnlook executable, for example, /usr/bin/svnlook or/opt/subversion/bin/svnlook.
   • RabbitMQ (amqp) Section
     • host — The hostname of the amqp server. Copy and paste the hostname from the Queue Server value from the Edit Source page in TeamForge EventQ.
     • port — The port number of the amqp server (default is 5672). Copy and paste the port number the Queue Server value from the Edit Source page in TeamForge EventQ.
     • username — Username used to access the amqp server. Copy and paste the queue username value from the Edit Source page in TeamForge EventQ.
     • password — Password used to access the amqp server. Copy and paste the queue password value from the Edit Source page in TeamForge EventQ.
   • TeamForge EventQ Section
     • source_association_key — key provided by TeamForge EventQ. Copy and paste the source association key value from the Edit Source page in TeamForge EventQ. For more on association kys, see Source Association Keys.

Activate Post-commit Hook

Copy or create a symlink called “post-commit” linking to post_commit_orc_publish_amqp.py.

Install a Subversion Adapter on Windows

Install a Subversion adapter as a post-commit hook on Windows.

Minimum Requirements:

• Subversion 1.7 or later.
• Python 2.6. Python v3.0 and later are not compatible.

Install Subversion Post-commit Hooks

Contact the CollabNet Support to download the eventQ-subversion-adapter-[version].zip adapter file and unpack it inside your repository’s hooks directory.

For example, if your repository’s hooks directory is located at C:\csvn\repositories\my_repo\hooks, cd C:\csvn\repositories\my_repo\hooks, download the zip file to the current location, then unzip eventQ-svn-adapter-1.0.zip.

This will unpack the following into your hooks directory:

• LICENSE.txt
• README.txt
• configdefaults.py
• configwrapper.py
• eventq_commit_gate_hook.py
• orchestrator_commit_gate_hook_snippet.sh
• pika/
• post-commit.bat.sample
• post_commit_orc_publish_amqp.conf.sample
• post_commit_orc_publish_amqp.py

Configuration

1. In a web browser, log into TeamForge EventQ as a privileged EventQ user and edit (or add) the relevant Commit source you wish to associate with this Subversion repository. Manage Sources > Toggle Commit > Add or Edit a Source > External repository
2. On the Edit Source page, locate the section titled “Adapter Configuration Information” and copy all values; these values are used later in the configuration process.
3. Switch back to your command line console and copy post_commit_orc_publish_amqp.conf.sample to post_commit_orc_publish_amqp.conf.
4. Edit post_commit_orc_publish_amqp.conf and modify options for your specific installation, for example:
   • Logging Section
     • filename — Full path to the desired log file, for example, C:\Temp\post-commit.log.
       Important: Ensure that the log file exists, or create it, before proceeding to the next step.
     • level — The logging level, which is one of these: debug , info, warning, error, or critical.
   • svnlook Section
     • svnlook — The path to your svnlook executable, for example, C:\csvn\bin\svnlook.exe.
   • RabbitMQ (amqp) Section
     • host — The hostname of the amqp server. Copy and paste the hostname from the Queue Server value from the Edit Source page in TeamForge EventQ.
     • port — The port number of the amqp server (default is 5672). Copy and paste the port number the Queue Server value from the Edit Source page in TeamForge EventQ.
     • username — Username used to access the amqp server. Copy and paste the queue username value from the Edit Source page in TeamForge EventQ.
     • password — Password used to access the amqp server. Copy and paste the queue password value from the Edit Source page in TeamForge EventQ.
       Note: A RabbitMQ server is optionally set up during the installation process. The installation process sets a default password for the eventq user and stores it for reference in the following file on the RabbitMQ host (root access required): /opt/collabnet/rabbitmq/.rabbit_passwords
   • TeamForge EventQ Section
     • source_association_key — key provided by TeamForge EventQ. Copy and paste the source association key value from the Edit Source page in TeamForge EventQ. For more on association kys, see Source Association Keys.

Activate Post-commit Hook

1. Copy the post-commit.bat.sample to post-commit.bat.
2. Edit the post-commit.bat for your specific installation: set PYTHON_EXE to the full path to python.exe executable. For CollabNet Subversion Edge, this is typically C:\csvn\Python25\python.exe.

Related Links

• Installing Pika
• Implementing Repository Hooks
• Subversion Adapter FAQs

Review Board Integration

The TeamForge EventQ-Review Board adapter enables communication between your Review Board server and TeamForge EventQ.

The Review Board adapter is packaged as a Review Board extension, and once installed provides web-based configuration inside Review Board.

The adapter supplies code review metadata for selected repositories to TeamForge EventQ. Code review metadata is associated to the proper source using source association keys (see Source Association Keys). Configuration therefore requires that the MQ server URL and credentials be supplied, along with the appropriate source association key.

Install or Upgrade a Review Board Adapter on Mac OS X/Linux

Install a Review Board adapter as a Review Board extension to retrieve code review data from instances of Review Board 1.7 and later.

Minimum Requirements

• Review Board 1.7 or higher
• Python 2.6 or 2.7. Version 3.0 and later are not compatible.
• setuptools 2.1 or later
• pip 1.5 - 6.0.6 (6.0.7 has issues and is not supported)

1. Upgrade pip. Upgrade pip if you do not have the minimum required version.

   sudo pip install --upgrade pip
   

2. Upgrade setuptools. Upgrade setuptools if you do not have the minimum required version.

   sudo pip install --upgrade setuptools
   

3. Install or upgrade the TeamForge EventQ-Review Board adapter as a Review Board extension.
   • Installation
     • If your server has internet access, use the command below to install the EventQ adapter as an extension.

       sudo pip install --use-wheel --no-index --find-links=http://eventq.collab.net/RHEL/6/x86_64/wheelhouse eventq_rb
       

     • If your server cannot access the internet, see Install a Review Board Adapter Without Internet Access instead.
   • Upgrade
     • Add the -U flag to upgrade to the latest version of the Review Board adapter.

       sudo pip install -U --use-wheel --no-index --find-links=http://eventq.collab.net/RHEL/6/x86_64/wheelhouse eventq_rb
       

4. Restart Review Board’s web server to register the newly installed EventQ adapter.

Install or Upgrade a Review Board Adapter Without Internet Access

The following instructions pertain to situations where your Review Board server does not have outbound access to the internet.

1. Contact the CollabNet Support to download the eventq_rb-[version].zip archive file.
2. Extract the archive file on the target server.

   unzip eventq_rb-[version].zip
   

3. Install or upgrade the TeamForge EventQ-Review Board adapter as a Review Board extension.
   • Use the command below to install the EventQ adapter as an extension.

     sudo pip install --use-wheel --no-index --find-links=PATH_TO_WHEELHOUSE eventq_rb
     

     Where PATH_TO_WHEELHOUSE is the path to the directory where you unzipped the adapter file. Suppose you unzipped the adapter file to the /tmp directory. In such a case, the PATH_TO_WHEELHOUSE is /tmp.

     Example:

     sudo pip install --use-wheel --no-index --find-links=/home/myusername/wheelhouse/ eventq_rb
     

   • Add the -U flag to upgrade to the latest version of the Review Board adapter.

     sudo pip install -U --use-wheel --no-index --find-links=PATH_TO_WHEELHOUSE eventq_rb
     

4. Restart Review Board’s web server to register the newly installed EventQ adapter.

Configure the Review Board Adapter (TeamForge Integrated)

Configure a TeamForge-integrated Review Board adapter to retrieve code review metadata.

Once the adapter has been installed successfully, configure Review Board by following these steps. The instructions below pertain to Review Board instances integrated with CollabNet TeamForge. See Configure the Review Board Adapter (Standalone) for configuration instructions when using a standalone version of Review Board.

Navigate to Review Board Extension Administration

1. Log into TeamForge as a site-admin user.
2. Select Admin > Integrated Apps.
3. Select the Review Board option and click Administer. You should now see Review Board administrative page.
4. Click Extensions from the menu bar to list your currently installed extensions. eventq-rb should be listed.
5. Click Enable to activate the EventQ extension.

6. Click Configure on the eventq-rb extension.

   The eventq-rb configuration page shows up. From here you can select and configure the repository whose reviews you would like to push to an EventQ review source. To create a review source in EventQ see: Manage Review Sources.

Configure a Review Board Respository

1. Select the repository from the Repository drop-down list.
2. Copy and paste the fields from the EventQ review source that you wish to associate to this repository.
3. Click Save.
4. Repeat steps 1 through 3 for each Review Board repository you wish to monitor in TeamForge EventQ.

Configure the Review Board Adapter (Standalone)

Configure a standalone instance of Review Board adapter to retrieve code review metadata.

Once the adapter has been installed successfully, configure Review Board by following these steps. The instructions below pertain to standalone Review Board instances.

Navigate to Review Board Extension Administration

1. Log into your Review Board instance as a privileged user.
2. Click the Admin link to reach the administrative page.
3. Click Extensions on the menu bar to list your currently installed extensions. eventq-rb should be listed.
4. Click Enable to activate the EventQ extension.

5. Click Configure on the eventq-rb extension.

   The eventq-rb configuration page shows up. From here you can select and configure the repository whose reviews you would like to push to an EventQ review source. To create a review source in EventQ see: Manage Review Sources.

Configure a Review Board Respository

1. Select the repository from the Repository drop-down list.
2. Copy and paste the fields from the EventQ review source that you wish to associate to this repository.
3. Click Save.
4. Repeat steps 1 through 3 for each Review Board repository you wish to monitor in TeamForge EventQ.