Private build slave/agent
A private build slave is needed because of Docker. See this quote from the FAQ:
This plugin is not available in Update Center as it requires docker on slave node. It's not possible to provide it on our infrastructure yet as it requires extended privileges to run.
The basic steps are described here.
In a nutshell what was done:
Create the Jenkins agent
- Define the agent through the "New Node" link in the left hand panel on this page.
- Click the hyperlink of the newly created agent and copy the command line from that page
- Add the proxy to that command (see this page) and wrap the command line in a service via
systemctl (this page has helpful information)
Add the public key to Jenkins
Go to the account settings for yona dev:
Go to the security keys page page:
Now add the public key using the add button.
Jenkins GitHub hook configuration
In order for Jenkins to be able to register its hooks in GitHub so that jobs can start after pushes you need to configure the user to connect. First go to the general URL to configure Jenkins: https://yonadev.ci.cloudbees.com/configure
In the GitHub section create the API token for the user (in our case the
yonabuild user). Go to the additional actions and choose "Convert login and password to token". This will create the proper API key and web hook in GitHub
Jenkins will return with a message like this: Created credentials with id dab87d22-5193-4303-92b6-ddfaea179c68 (can use it for GitHub Server Config).
Important: you need to refresh the page! Now you can also delete this server configuration.
Now you can choose to use these credentials in the plugin:
Register Jenkins as app on GitHub
Go to the newly created app (https://github.com/settings/apps/yona-jenkins) and set the logo:
Generate a private key
Convert the key file:
Install the app:
For all repositories:
Go to the Jenkins credentials and add a credential for the app on GitHub:
Click "Test Connection" to verify everything is setup correctly, and if so, click OK.
Jenkins job configurations
Build of master branch
The Yona server project has been setup to do automatic integration testing whenever a commit is done to the master. The build is implemented as a straight-forward pipeline (see the script):
- Build the JARs
- Run the unit tests
- Build Docker images
- Push them to Docker Hub
- Setup test server
- Pull in the Docker images
- Set up a clean database
- Load the database schema
- Create fresh Docker images from the new containers
- Run integration tests
This pipeline is triggered on a commit to the the master branch at GitHub. A webhook is configured on GitHub that notifies the Jenkins server about new commits. The job
build-and-test/master is triggered for every commit to the master. See Build of master branch. After the integration tests have finished we will know if the changes are good and if the branch is stable.
The job configuration contains a couple of interesting parts.
The first step is to configure where the branch(es) to be built are stored. This is defined through the GitHub plug-in (so not the Git plug-in!):
With this, the build job will scan what branches contain a Jenkins file (in our case, this scanning is limited to just the master branch, through the "Include branches" setting) and the build will automatically trigger when an update is pushed to that branch.
All other settings are left to their defaults:
The pipeline script makes use of a few credentials:
Check of pull requests
To verify pull requests, a separate build job is defined, based on the GitHub pull request builder plug-in.
The plug-in configuration (Jenkins → Manage Jenkins → Configure system) looks as follows:
The shared secret is a long random string that is to be entered in the GitHub webhook configuration.
GitHub webhook configuration
To trigger the build on creation of a new pull request or when a comment is added to an existing one, a webhook needs to be added to the YonaServer repository, through this URL. That shows a screen like this:
The new webhook is defined as follows:
This job is called
Server - pull request check. There are a number of noteworthy aspects in the configuration of the job.
CloudBees DEV@cloud Authorization
The GitHub project property needs to be set. If not, the plug-in will log a
NullPointerException. We are restricting the job to our regular build executor for server builds.
Source code management
The respository URL needs to be specified again. Note that it's slightly different from the previous screen. The Refspec (default hidden behind the Advanced button) and the Branch Specifier are noteworthy. The Refspec used below indicates that we want want to build pull requests only, not branches. The Branch Specifier indicates that we want to build the pull request, not the actual commit.
Note that Credentials is intentionally set to None, as this property isn't used.
Here we select the GitHub Pull Request builder. The admin list (the people who can manage the pull request building through special comments on the pull request, as described here) can be managed here. The "github hooks" box is checked, so Jenkins does not poll GitHub but responds to a webhook trigger from GitHub.
Binds the GitHub API token to the environment variable
GITHUB_TOKEN, for later use with SonarQube:
The build consists of a few steps:
gradle check and a run of the SonarQube scanner:
We gather the test results, to the Build of master branch.
Building a Helm package
One of the steps in the pipeline includes building a Helm package. The logic to build the package is implemented in publish-helm-package.sh. Here we describe the steps that are required to prepare the build slave to build and publish Helm packages.
- Install Helm
- Create working directory
- Initialize Helm (client-only):
helm init --home /opt/ope-cloudbees/yona/helm/.helm --client-only
- Add the Yona repo:
helm repo add --home /opt/ope-cloudbees/yona/helm/.helm yona
- Clone Helm charts repository into it: