This page should be used to collect ideas to shape Kieker's delivery pipeline

Current Situation (for the Kieker core)

...

1. job: http://kieker.uni-kiel.de/jenkins/job/kieker/

• runs

  Code Block
  ./gradlew.sh jenkinsBuild --stacktrace

...

Table of Contents

Pipelines

Our current delivery pipeline is using infrastructure as code using a declarative Jenkinsfile that is parsed by Jenkins upon receiving a manual trigger or a webhook from GitHub.

Build Pipeline

The current Jenkinsfile in the 'stable' branch can be found at: https://github.com/kieker-monitoring/kieker/blob/stable/Jenkinsfile

The figure below describes the most important aspects of the build steps (attached, you can find the figure as svg and source file (dia)):

Image Added

Pull Request (PR)

Whenever a branch should be integrated into the main development branch of Kieker, it is required to create a pull request towards the 'master' branch. To get a branch merged into the 'master', all automated checks (i.e., branch build, PR build) need to be successful and there needs to be at least one positive review by another developer. If the requirements are met, one of the Kieker developers can merge the branch into 'master'.

From 'master' to 'stable'

Whenever a commit is made to the 'master' branch (e.g., due to a PR towards 'master'), a new build is triggered in Jenkins. This build executes all build steps again to ensure that all requirements are met. If this is the case, the last two steps of that build push the current state in 'master' to the 'stable' branch and upload a snapshot of the artifacts to MavenCentral.

Feature branch

A pipeline for a branch is instantiated for each branch with a prefix KIEKER- (e.g., KIEKER-1432) and triggered for each subsequent commit.

Stages: Same as 'master' to 'stable' except for the final push.

Nightly release

Executed each night based on the stable branch. 

Stages: Corresponds to Master to stable followed by the micro-benchmark.

Nightly release artifacts are available via https://build.se.informatik.uni-kiel.de/jenkins/job/kieker-nightly-release/

...

runs

./gradlew.sh jenkinsNightlyBuild

chmod +x kieker-examples/OverheadEvaluationMicrobenchmark/executeRemoteMicroBenchmark.sh
kieker-examples/OverheadEvaluationMicrobenchmark/executeRemoteMicroBenchmark.sh

...

lastSuccessfulBuild/artifact/

Pipeline on Jenkins: https://build.se.informatik.uni-kiel.de/jenkins/job/kieker-nightly-

...

1. runs

   Code Block
   ./gradlew.sh jenkinsBuild --stacktrace

...

1. runs

   Code Block
   ./gradlew.sh jenkinsNightlyBuild

...

release/

Release

Executed (manually) to create release candidates and, finally the release, from the respective release branch.

Stages: Corresponds to Master to stable

Kieker Build Docker Containers

The repository containing all the Dockerfile files: https://github.com/kieker-monitoring-docker/kieker-build

The DockerHub containing the automatically built containers: https://hub.docker.com/r/kieker/kieker-build/

The most important tags right now are:

• kieker/kieker-build:base (Base image for all other containers based on Ubuntu 15.10 and the additional R dependencies)
• kieker/kieker-build:openjdk8 (Based on :base with openjdk-8 installed)

Some Todos

Pipeline template not only for the Kieker core but also for other projects in the "Kieker orbit"

Stages

• Build
  • Compiles, creates the release artifacts, runs the (fast) unit tests, maybe some smoke tests + static code analysis
• Integration Test
  • Runs the more expensive integration tests
• User Acceptance Test

Pipelines

Pull Request Check

• Goal: Makes sure that a pull request can be merged without conflict, can be compiled, passes the unit tests + static code analysis check and (more? we probably want to exclude the benchmarking here)
• Trigger: Pull request
• Executed stages: build

Integration

Nightly Release

Release

Possible Implementations

Jenkins

Travis CI

...

•  Test with different versions of Java (6?, 7, 8)
•  No artifact is being rebuild. In each stage after the build stage, the build artifacts from the build stage are used.

...

•  Does it make sense to have a stable branch in addition to master into which only those changesets are merged that passed the entire pipeline?
•  Do we want to have a gatekeeper condition (threshold) on the regression benchmark?
•   Separate the unit tests from the integration tests

•  Insert the call of the check-script (in the stage around the acceptance test?)
  
•  Merge the Gradle threshold code into master to be able to have quality gates for code analysis
•  Think about a way for continuous build numbering without breaking sonatype/maven conventions (we could use the build numbering in snap-ci to do so)