Upgrade TeamForge on Same Hardware with All Services on a Single Server

In this single server setup, the following TeamForge services run on the TeamForge Application Server (server-01).

• TeamForge Application Server (ctfcore)
• Database Server (ctfcore-database and ctfcore-datamart)
• Codesearch Server (codesearch)
• Mail Server (mail)
• ETL Server (etl)
• Git Integration Server (gerrit and gerrit-database)
• SCM Integration Server (subversion and cvs)
• Search Server (search).
• TeamForge EventQ Server (eventq, mongodb, redis and rabbitmq)
• TeamForge CLI Server (cliserver)
• Review Board (reviewboard, reviewboard-database, reviewboard-adapter)
• CLI Server (cliserver)
• TeamForge Baseline (baseline, baseline-database, baseline-post-install)¹
• TeamForge Webhooks-based Event Broker (webr webr-database)²

Dos and Don’ts

chmod -R 775 $SUBVERSION_REPOSITORY_BASE

The following instructions are valid for both RHEL/CentOS 6.10/7.6 platforms. Specific steps, if applicable only for a particular RHEL/CentOS platform, are called out explicitly.

No backup is required for same hardware upgrades. However, you can create a backup as a precaution. See Back up and Restore TeamForge Database, Data Directories and site-options.conf.

Uninstall Custom Event Handlers, Hot Fixes, Add-ons and Review Board

1. Log on to the TeamForge Application Server (server-01).
2. SOAP 50 is no longer supported. Back up all your custom event handlers and remove all the event handler JAR files before starting your TeamForge upgrade process.
   1. Go to My Workspace > Admin.
   2. Click System Tools from the Projects menu.
   3. Click Customizations.
   4. Select the custom event handler and click Delete.
      Tip: Post upgrade, you can add custom event handlers again from the backup while making sure that you don’t have SOAP50 (deprecated) library used.
3. Uninstall hotfixes and add-ons, if any, installed on your site.

Create a Database Dump

TeamForge 19.0 supports PostgreSQL 11.1. If you are upgrading from TeamForge 17.1 or earlier versions, PostgreSQL 9.2/9.3 data cannot be migrated automatically. To migrate manually:

• Create a dump of your existing database.
• Stop the site.
• Remove the PostgreSQL cluster in /var/lib/pgsql/9.x.
• Run ‘teamforge deploy’.
• Restore your database by loading the dumps.
• Run the ‘teamforge provision’ command to do the migration and start the site.

1. Create a dump of your site’s database if and only if you are upgrading from TeamForge 17.1 or earlier versions.

   su - postgres
   /usr/bin/pg_dumpall > /opt/collabnet/teamforge/var/pgsql/9.3/backups/teamforge_data_backup.dmp
   exit
   mkdir /tmp/backup_dir
   cp /opt/collabnet/teamforge/var/pgsql/9.3/backups/teamforge_data_backup.dmp /tmp/backup_dir/
   

2. Remove the PostgreSQL cluster in /var/lib/pgsql/9.3.

   mv /var/lib/pgsql/9.3 /var/lib/pgsql/9.3_old
   

yum upgrade

1. Stop TeamForge.

   • If you are upgrading from TeamForge 16.7 or earlier releases:

     /etc/init.d/collabnet stop
     

   • If you are upgrading from TeamForge 16.10, 17.1, or 17.4 releases:

     /opt/collabnet/teamforge/bin/teamforge stop
     

   • If you are upgrading from TeamForge 17.8 or later releases:

     teamforge stop
     

2. Stop EventQ.

   • If you are upgrading from TeamForge 16.3:

     /etc/init.d/orchestrate stop
     

   • If you are upgrading from TeamForge 16.7, 16.10, or 17.1 release:

     /etc/init.d/eventq stop
     /etc/init.d/collabnet-rabbitmq-server stop
     /etc/init.d/collabnet-mongod stop
     

   • If you are upgrading from TeamForge 17.4 release:

     /opt/collabnet/teamforge/bin/teamforge stop
     

   • If you are upgrading from TeamForge 17.8 or later releases:

     teamforge stop
     

3. Upgrade the operating system packages.

   yum upgrade
   

Configure the TeamForge Installation Repository

1. Configure the TeamForge installation repository.

   TeamForge Installation Repository Configuration for Sites with Internet Access

   1. Contact the CollabNet Support and download the TeamForge 19.0 installation repository package to /tmp.
   2. Install the repository package.

      yum install -y /tmp/collabnet-teamforge-repo-19.0-0-noarch.rpm
      

   3. Refresh your repository cache.

      yum clean all
      

   TeamForge Installation Repository Configuration for Sites without Internet Access

   1. Contact the CollabNet Support to get the auxiliary installer package for TeamForge 19.0 disconnected installation and save it in /tmp.
      • RHEL/CentOS 6.10 64 bit: CTF-Disconnected-media-19.0.573-1320.rhel6.x86_64.rpm
      • RHEL/CentOS 7.6 64 bit: CTF-Disconnected-media-19.0.573-1320.rhel7.x86_64.rpm
      • In addition to the above CentOS 7.6 64 bit RPM package, you must get the following CentOS 7.6 compatibility RPM, which is required for TeamForge 19.0 disconnected media installation on CentOS 7.6 profile: compat-ctf-dc-media-1.2-1.el7.noarch.rpm.
   2. Unpack the disconnected installation package.

      rpm -Uvh <package-name>
      

   3. Unpack the compat-ctf-dc-media-1.2-1.el7.noarch.rpm package if you are installing TeamForge 19.0 on CentOS 7.6.

      rpm -ivh compat-ctf-dc-media-1.2-1.el7.noarch.rpm
      

   4. If not mounted already, mount the RHEL/CentOS installation DVD.

      The DVD contains the necessary software and utilities required for installing TeamForge without internet access. In the following commands, replace “cdrom” with the identifier for your server’s CD/DVD drive, if necessary.

      cd /media/
      mkdir cdrom
      mount /dev/cdrom ./cdrom/
      

      If there are any spaces in the automount, unmount it first and mount it as a filepath, with no spaces.

   5. Create a yum configuration file that points to the RHEL/CentOS installation DVD.

      vi /etc/yum.repos.d/cdrom.repo
      

      Here’s a sample yum configuration file.

      [RHEL-CDROM]
      name=RHEL CDRom 			
      baseurl=file:///media/cdrom/Server/
      gpgfile=file:///media/cdrom/RPM-GPG-KEY-redhat-release 
      enabled=1
      gpgcheck=0
      

   6. Verify your yum configuration files.

      yum list httpd
      yum list apr
      

Upgrade the TeamForge Services

1. Upgrade TeamForge.

   yum install teamforge
   

   Install the Baseline packages.

   yum install teamforge-baseline
   

   Install the TeamForge Webhooks-based Event Broker.

   yum install teamforge-webr
   

2. Run the following command to upgrade the Binary application packages. This is required if and only if you are upgrading from TeamForge 16.10 (or earlier) to TeamForge 17.8 (or later).

   yum install cn-binary
   

Set up the site-options.conf File

1. Set up the site-options.conf file.
   Important: See Site options change log for a list of site option changes. While upgrading to a latest TeamForge release, make sure that obsolete site option tokens, if any, are removed from the site-options.conf file of the TeamForge version you are upgrading to.

   vi /opt/collabnet/teamforge/etc/site-options.conf
   

   host:SERVICES Token

   server-01:SERVICES = ctfcore ctfcore-database ctfcore-datamart mail etl search codesearch subversion cvs eventq redis mongodb rabbitmq cliserver gerrit gerrit-database binary binary-database reviewboard reviewboard-database reviewboard-adapter baseline baseline-database baseline-post-install webr webr-database
   

   host:PUBLIC_FQDN Token

   server-01:PUBLIC_FQDN = my.app.domain.com
   

   Note: You cannot have a separate PUBLIC_FQDN for EventQ.

   MONGODB_APP_DATABASE_NAME

   Set the MONGODB_APP_DATABASE_NAME token with EventQ’s database name in the site-options.conf file. Do this if and only if you are upgrading from TeamForge 17.1 or earlier to TeamForge 17.4 or later.

   MONGODB_APP_DATABASE_NAME=orchestrate
   

   Save the site-options.conf file.

   For further customization of your site configuration (SSL settings, password policy settings, PostgreSQL settings, LDAP settings and so on): Site options configuration contd…

   SSL Tokens

   SSL is enabled by default and a self-signed certificate is auto-generated. Use the following tokens to adjust this behavior.

   SSL_CERT_FILE=
   SSL_KEY_FILE=
   SSL_CHAIN_FILE=
   

   • To generate the SSL certificates, see Generate SSL certificates.
   • Have the custom SSL certificate and private key for custom SSL certificate in place and provide their absolute paths in these tokens. SSL_CHAIN_FILE (intermediate certificate) is optional.
   • You can also encrypt the data traffic between the application and database servers and between the ETL and datamart servers in a distributed setup. Use the [DATABASE_SSL][siteoptiontokens.html#DATABASE_SSL] token to do that. See Encrypt Database Network Traffic.

   Password Tokens

   • TeamForge 7.1 and later support automatic password creation. See AUTO_DATA for more information.

   • Set the REQUIRE_PASSWORD_SECURITY token to true to enforce password security policy for the site.

     If the token REQUIRE_PASSWORD_SECURITY is enabled, then set a value for the token, PASSWORD_CONTROL_EFFECTIVE_DATE.

     Warning: The Password Control Kit (PCK) disables, deletes or expires user accounts that don’t meet the password security requirements starting from the date set for the PASSWORD_CONTROL_EFFECTIVE_DATE token. If a date is not set, the PCK disables, deletes or expires user accounts immediately. See PASSWORD_CONTROL_EFFECTIVE_DATE for more information.

     You can also set the following tokens to enforce a more stricter password policy:

     • MINIMUM_PASSWORD_LENGTH
     • MAX_PASSWORD_LENGTH
     • PASSWORD_REQUIRES_NUMBER
     • PASSWORD_REQUIRES_NON_ALPHANUM
     • PASSWORD_REQUIRES_MIXED_CASE
     • REQUIRE_PASSWORD_SECURITY
     • LOGIN_ATTEMPT_LOCK
     • PASSWORD_HISTORY_AGE
     • ALLOW_PASSWORD_DICTIONARY_WORD
   • If the token REQUIRE_RANDOM_ADMIN_PASSWORD is already set to true, then set the token ADMIN_EMAIL with a valid email address.

     ADMIN_EMAIL=root@{__APPLICATION_HOST__}
     

   • If you have LDAP set up for external authentication, you must set the REQUIRE_USER_PASSWORD_CHANGE site options token to false.
   • Verify and update the list of non-expiring TeamForge user accounts (password never expires).

     USERS_WITH_NO_EXPIRY_PASSWORD=admin,nobody,system,scmviewer,scmadmin
     

   Prevent Cross-site Scripting

   An attacker could potentially upload an HTML page to TeamForge that contains active code, such as JavaScript. This active code would then be executed by clients’ browsers when they view the page, which can harm the system.

   To prevent an attack of this sort, you can specify whether or not HTML code is displayed in TeamForge. This flag applies to all documents, tracker, task, and forum attachments, and files in the file release system.

   Set the SAFE_DOWNLOAD_MODE token according to your requirements. For more information, see SAFE_DOWNLOAD_MODE.

   PostgreSQL Tokens and Settings

   Make sure the PostgreSQL tokens in the site-options.conf file are set as recommended in the following topic: What are the right PostgreSQL settings for my site?

   JAVA_OPTS

   Configure the JBOSS_JAVA_OPTS site-options.conf token. See JBOSS_JAVA_OPTS.

   Note: All JVM parameters but -Xms1024m and -Xmx2048m have been hard-coded in the TeamForge core application. You need not manually configure any other parameter (such as -XX:MaxMetaspaceSize=512m -XX:ReservedCodeCacheSize=128M -server -XX:+HeapDumpOnOutOfMemoryError -Djsse.enableSNIExtension=false -Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000) in the site-options.conf file.

   TeamForge 18.1 (and later) supports Java 9. As a result of changes to the logging framework in Java 9, the PrintGCDetails and PrintGCTimeStamps logging options are no longer supported. Remove these options from the following tokens while upgrading to TeamForge 18.1 or later.

   • JBOSS_JAVA_OPTS
   • PHOENIX_JAVA_OPTS
   • INTEGRATION_JAVA_OPTS
   • ETL_JAVA_OPTS
   • ELASTICSEARCH_JAVA_OPTS

   TeamForge provision fails on sites that use these options post upgrade to TeamForge 18.1.

   Save the site-options.conf file.

2. Deploy TeamForge and reload the PostgreSQL dump. Do this if and only if you are upgrading from TeamForge 17.1 or earlier versions (with PostgreSQL 9.3/9.2).

   Warning: You must move the PostgreSQL 9.x directory (mv /var/lib/pgsql/9.x /var/lib/pgsql/9.x_old) after reloading the database dump, failing which the teamforge provision command will not be successful.

   teamforge deploy
   

   su - postgres 
   psql < /tmp/backup_dir/teamforge_data_backup.dmp (for TeamForge database)
   psql -p 5632 < /tmp/teamforge_reportsdata_backup.dmp (for reporting database)
   

Provision Services

1. Do this if and only if you have EventQ integration and are upgrading from TeamForge 17.1 or earlier to TeamForge 19.0.

   Copy the /opt/collabnet/rabbitmq/var and /opt/collabnet/mongodb/data directories to /opt/collabnet/teamforge/var/rabbitmq and /opt/collabnet/teamforge/var/mongodb directories respectively.

   cp -R /opt/collabnet/rabbitmq/var /opt/collabnet/teamforge/var/rabbitmq
   cp -R /opt/collabnet/mongodb/data /opt/collabnet/teamforge/var/mongodb
   

2. Provision services.

   teamforge provision
   

   TeamForge 19.0 installer expects the system locale to be LANG=en_US.UTF-8. TeamForge create runtime (teamforge provision) fails otherwise.

Finishing Tasks

1. Run the /var/lib/pgsql/analyze_new_cluster.sh script. This is required if and only if you are upgrading from TeamForge 17.1 (or earlier) to TeamForge 17.8 (or later).

   su - postgres -c "/var/lib/pgsql/analyze_new_cluster.sh"
   

2. If you have CVS integrations, synchronize permissions post upgrade. See Synchronize TeamForge Source Control Integrations.
3. Verify TeamForge upgrade.
   1. Reboot the server and make sure all services come up automatically at startup.
   2. Log on to the TeamForge web application using the default Admin credentials.
      • Username: admin
      • Password: admin
   3. If your site has custom branding, verify that your branding changes still work as intended. See Customize TeamForge.
   4. Let your site’s users know they’ve been upgraded. See Create a Site-wide Broadcast.

Post Upgrade Tasks

• Run TeamForge in SELinux enabled Mode
• Add EventQ to Existing Projects
• Users are not getting email notifications for review requests and reviews. What should I do?
• Integrate using TeamForge Webhooks-based Event Broker (Jenkins and JIRA)

Also See…

• FAQs on Install / Upgrade / Administration
• TeamForge upgrade fails when migrating Baseline database to the latest schema. What should I do?