Skip to end of metadata
Go to start of metadata

Introduction

The build server for Yona is hosted at CloudBees, under the FOSS programhttps://yonadev.ci.cloudbees.com/. This page describes the configuration of the build server.

Private build slave/agent

A private build slave is needed because of Docker. See this quote from the FAQ:

This plugin is not available in Update Center as it requires docker on slave node. It's not possible to provide it on our infrastructure yet as it requires extended privileges to run.

The basic steps are described here.

In a nutshell what was done:

Create the Jenkins agent

  • Define the agent through the "New Node" link in the left hand panel on this page.
  • Click the hyperlink of the newly created agent and copy the command line from that page
  • Add the proxy to that command (see this page) and wrap the command line in a service via systemctl (this page has helpful information)

Relevant commands:

# Get last 150 log records
journalctl -u ope.service | tail -150
# Restart the OPE
systemctl restart ope.service

Add the public key to Jenkins

Go to the account settings for yona dev:

Go to the security keys page page:

Now add the public key using the add button.

Jenkins GitHub hook configuration

In order for Jenkins to be able to register its hooks in GitHub so that jobs can start after pushes you need to configure the user to connect. First go to the general URL to configure Jenkins: https://yonadev.ci.cloudbees.com/configure

In the GitHub section create the API token for the user (in our case the yonabuild user). Go to the additional actions and choose "Convert login and password to token". This will create the proper API key and web hook in GitHub

Jenkins will return with a message like this: Created credentials with id dab87d22-5193-4303-92b6-ddfaea179c68 (can use it for GitHub Server Config)

Important: you need to refresh the page! Now you can also delete this server configuration.

Now you can choose to use these credentials in the plugin:

Register Jenkins as app on GitHub

See this page and this one. Register the Jenkins app through this link and fill out the form as follows:

Go to the newly created app (https://github.com/settings/apps/yona-jenkins) and set the logo:

Generate a private key

Generate private key

Convert the key file:

openssl pkcs8 -topk8 -inform PEM -outform PEM -in yona-jenkins.2020-05-22.private-key.pem -out converted-github-app.pem -nocrypt

Install the app:

For all repositories:

Go to the Jenkins credentials and add a credential for the app on GitHub:

Click "Test Connection" to verify everything is setup correctly, and if so, click OK.

Jenkins job configurations

Build of master branch

The Yona server project has been setup to do automatic integration testing whenever a commit is done to the master. The build is implemented as a straight-forward pipeline (see the script):

  1. Build
    1. Build the JARs
    2. Run the unit tests
    3. Build Docker images
    4. Push them to Docker Hub
  2. Setup test server
    1. Pull in the Docker images
    2. Set up a clean database
    3. Load the database schema
    4. Create fresh Docker images from the new containers
  3. Run integration tests

This pipeline is triggered on a commit to the the master branch at GitHub. A webhook is configured on GitHub that notifies the Jenkins server about new commits. The job build-and-test/master is triggered for every commit to the master. See Build of master branch. After the integration tests have finished we will know if the changes are good and if the branch is stable.

Job configuration

The job configuration contains a couple of interesting parts. 

Branch sources

The first step is to configure where the branch(es) to be built are stored. This is defined through the GitHub plug-in (so not the Git plug-in!):

With this, the build job will scan what branches contain a Jenkins file (in our case, this scanning is limited to just the master branch, through the "Include branches" setting) and the build will automatically trigger when an update is pushed to that branch.

Other settings

All other settings are left to their defaults:

Credentials

The pipeline script makes use of a few credentials:

Check of pull requests

To verify pull requests, a separate build job is defined, based on the GitHub pull request builder plug-in.

Plug-in configuration

The plug-in configuration (Jenkins → Manage Jenkins → Configure system) looks as follows:

The shared secret is a long random string that is to be entered in the GitHub webhook configuration.

GitHub webhook configuration

To trigger the build on creation of a new pull request or when a comment is added to an existing one, a webhook needs to be added to the YonaServer repository, through this URL. That shows a screen like this:

The new webhook is defined as follows:

Job configuration

This job is called Server - pull request check. There are a number of noteworthy aspects in the configuration of the job.

CloudBees DEV@cloud Authorization

The GitHub project property needs to be set. If not, the plug-in will log a NullPointerException. We are restricting the job to our regular build executor for server builds.

Source code management

The respository URL needs to be specified again. Note that it's slightly different from the previous screen. The Refspec (default hidden behind the Advanced button) and the Branch Specifier are noteworthy. The Refspec used below indicates that we want want to build pull requests only, not branches. The Branch Specifier indicates that we want to build the pull request, not the actual commit.

Note that Credentials is intentionally set to None, as this property isn't used.

Build triggers

Here we select the GitHub Pull Request builder. The admin list (the people who can manage the pull request building through special comments on the pull request, as described here) can be managed here. The "github hooks" box is checked, so Jenkins does not poll GitHub but responds to a webhook trigger from GitHub.

Bindings

Binds the GitHub API token to the environment variable GITHUB_TOKEN, for later use with SonarQube:

Build

The build consists of a few steps: gradle cleangradle check and a run of the SonarQube scanner:

Post-build steps

We gather the test results, to the Build of master branch.

Building a Helm package

One of the steps in the pipeline includes building a Helm package. The logic to build the package is implemented in publish-helm-package.sh. Here we describe the steps that are required to prepare the build slave to build and publish Helm packages.

  1. Install Helm
  2. Create working directory /opt/ope-cloudbees/yona/helm
  3. Initialize Helm (client-only): helm init --home /opt/ope-cloudbees/yona/helm/.helm --client-only
  4. Add the Yona repo: helm repo add --home /opt/ope-cloudbees/yona/helm/.helm yona https://yonadev.github.io/helm-charts
  5. Clone Helm charts repository into it: git clone https://github.com/yonadev/helm-charts.git
  • No labels