This page is now hosted on https://odp.finos.org/docs/development-infrastructure/continuous-integration/intro

Continuous Integration (CI) allows developers to run different type of validation processes automatically at commit / merge time, in order to identify problems earlier. As a result, configuring CI is inherently language and eco-system specific.

Travis CI

Travis CI is a hosted, distributed continuous integration service used to build and test software projects hosted at GitHub.

To enable Travis CI on a Foundation hosted project:

Sign in to Travis CI with your Github credentials
Access the Travis CI profile and sync your project
Drop a .travis.yml file in the root folder of your GitHub repo (check this .travis.yml as example)
1. (optional) If you need to perform variable encryption and other useful commands, install the Travis CI Client (using gem install travis)
Access https://travis-ci.org/<program name>/<repository-name> and validate the build

Please check the Travis supported languages and how to customize your build to understand if your language is supported and how to configure additional build features.

A badge can be added at the top of the project's root-level README.md file, using the following Markdown syntax:

[![Build Status](https://travis-ci.org/<program name>/<repository name>.svg)](https://travis-ci.org/<program name>/<repository name>)

This will appear as follows and link to the TravisCI build status page for that repository.

Continuous (artifact) deployment in Java

When using Java, Travis CI can be easily configured to publish new snapshot artifacts on commit; to enable this feature, a project committer can follow these simple steps:

Follow the Java Snapshot deployment configuration ; as a result, you should have username/password credentials (mentioned below as CI_DEPLOY_USERNAME and CI_DEPLOY_PASSWORD) to access issues.sonatype.org
Raise a HELP issue or send an email to help@finos.org to request permission to deploy artifacts on Maven remote repositories; this action is not required if the developer has already been granted access previously

Commit a settings.xml file in the project root folder

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>ossrh</id>
      <username>${env.CI_DEPLOY_USERNAME}</username>
      <password>${env.CI_DEPLOY_PASSWORD}</password>
    </server>
  </servers>
</settings>

Define CI_DEPLOY_USERNAME and CI_DEPLOY_PASSWORD variables with Travis CI; make sure that they're encrypted and hidden during the build process; the credentials to use are the ones defined on step 1; if you don't want to share your username/password credentials, you can request and use Nexus tokens
Change the mvn build command to
1. Invoke the deploy goal
2. Append --settings settings.xml at the end of the build command

This tutorial walks through the mentioned steps, with more details and configuration options; a working example of this configuration is the symphony-java-sample-bots project.

Continuous release in Python

Travis CI can be configured to use python-semantic-release, a package that simplifies and automates versioning for python projects; a project committer can raise a HELP issue or send an email to help@finos.org (following the template below) to request Foundation staff to apply the proper Travis CI project settings; packages will be published on behalf of the Foundation's pypi user:

h4. Project coordinates
<_GitHub repository url and other useful info_>

h4. Previously published packages
<_Links to previously published packages on pypi.python.org (or any other publicly available repository_>

Continuous release in NodeJS

Travis CI can easily publish packages to npm using semantic-release.org, which delegates release operations to your CI environment and allows you to control versioning using commits (commitizen); as a result, npm packages will be continuously released on each code repository change.

Follow our instructions on how to register a package and user in the Foundation npm organisation.

Example of Configuring Integration Testing

In order to run this sample (some sample bots from the Symphony program), the Travis build needs to be configured to add the following items:

before_script: "curl -s https://raw.githubusercontent.com/symphonyoss/contrib-toolbox/master/scripts/download-files.sh | bash"

env:
 global:
  - FOUNDATION_API_URL=https://foundation-dev-api.symphony.com
  - FOUNDATION_POD_URL=https://foundation-dev.symphony.com
  - SESSIONAUTH_URL=$FOUNDATION_API_URL/sessionauth
  - KEYAUTH_URL=$FOUNDATION_API_URL/keyauth
  - POD_URL=$FOUNDATION_POD_URL/pod
  - AGENT_URL=$FOUNDATION_POD_URL/agent

At this point, the certificates are in place and downloaded into the build box; integration tests can resolve User Identity certificates by accessing environment variables.

The symphony-java-sample-bots was amongst the first Foundation projects that was enabled to run integration tests from Travis CI; the Travis build logs for this project show two interesting items:

The (hidden) environment variables that identify User Identity certificates (in this case 2, one for the message sender, one for the message receiver)

Setting environment variables from repository settings
$ export DOWNLOAD_PATH=[secure]
$ export DOWNLOAD_ITEMS=[secure]
$ export TRUSTSTORE_FILE=[secure]
$ export TRUSTSTORE_PASSWORD=[secure]
$ export BOT_USER_EMAIL=[secure]
$ export BOT_USER_CERT_FILE=[secure]
$ export SENDER_USER_EMAIL=[secure]
$ export SENDER_USER_CERT_FILE=[secure]
$ export DOWNLOAD_HOST=[secure]
$ export SENDER_USER_CERT_PASSWORD=[secure]
$ export BOT_USER_CERT_PASSWORD=[secure]

The integration test logs of a successful run

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.symphonyoss.simplebot.EchoBotIT
chat initialised
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.132 sec - in org.symphonyoss.simplebot.EchoBotIT

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Known Certificate Issues

Caused by: javax.crypto.BadPaddingException: Given final block not properly padded at
org.symphonyoss.simplebot.EchoBotIT.sendAndReceiveEcho(EchoBotIT.java:64)

This basically means that the certificate password is wrong; to validate the certificate using the following command:

openssl pkcs12 -info -nodes \
-in <certificate_path> \
-passin pass:<certificate_password>

Reminder for Foundation staff: every password symbol must be escaped by prefixing it with character ‘\’, before setting it as Travis environment variable.

Branch specific build commands

Travis CI does not provide a syntax to specify branch-specific build commands; however, it is possible to use the following .travis.yml syntax to workaround it; in this example, the Maven build will use the versioneye profile only if the current branch is master.

script:
- "[[ $TRAVIS_BRANCH =~ master ]] && mvn clean deploy -Pintegration-testing,versioneye --settings settings.xml || true"
- "[[ $TRAVIS_BRANCH =~ dev ]] && mvn clean deploy -Pintegration-testing --settings settings.xml || true"

MyGet

Myget is a friction-free continuous integration & delivery for your nuget, npm, bower and vsix packages; the Foundation provides package repositories, which allow:

Pre-release build and publishing, using build environments that are equipped with .NET Framework and Visual Studio (headless); it also features source code tagging and version updates on source code based on incremental build number, to fully automate the package publishing pipeline
Sync with NuGet, that can be either manual (promoting pre-releases to releases using MyGet web interface) or automatic (a committer must create a personal account on MyGet and request access from the Foundation in order to be able to push packages)

In order to request a project to be integrated with MyGet, a project committer can sign up to MyGet and open a HELP issue or send an email to help@finos.org, using the following the template:

h4. Project coordinates
<_GitHub project url and other useful info_>

h4. MyGet username
<_your MyGet username_>

h4. CSProj and CS files
<_Where project descriptors are located_>

h4. Packages
<_A list of the packages that are published; by default all packages are taken into account_>

h4. Publishing strategy to NuGet
<_ Whether to enable automatic publishing based on source code branching or rely on manual package pushing using the MyGet interface; if the latter, a MyGet personal account username must be provided _>

An example of project building with MyGet is the RestApiClient project in the Symphony program, which is also published on the ssf-feed MyGet repository and NuGet.

Badges can be added at the top of the project's root-level README.md file, using the following Markdown syntax:

Build status:

[![MyGet Build Status](https://www.myget.org/BuildSource/Badge/<feed name>?identifier=<guid>)](https://www.myget.org/feed/<feed name>/package/nuget/<package name>)

Latest Pre-Release package published:

[![MyGet Pre Release](https://img.shields.io/myget/<feed name>/v/<package name>.svg)](https://www.myget.org/feed/<feed name>/package/nuget/<package name>)

Integration testing against the Foundation's Symphony Pod

A Continuous Integration environment can be integrated to run integration tests against the Symphony Pod in the Open Developer Platform, in order to provide an automated end-to-end testing process.

Requesting CI integration

Any committer can open a HELP issue or send an email to help@finos.org to request enabling CI on a hosted project. Please follow this template:

h4. Project coordinates
<_GitHub project url and other useful info_>

h4. Integration tests
<_List and describe (what to expect, how long it takes to run, the integration tests that you want to run in a CI environment_>

h4. Local run
<_Describe how to setup a local environment and run the integration tests_>

h4. Environment variables
<_List all environment variables needed to run the integration tests_>

h4. Current environment
<_Describe the current build environment, if any_>

The Foundation staff will configure all build variables that are needed to download the User Identity certificates (.p12 files) and run the integration tests.