Build your own Jenkins! Introducing Custom WAR/Docker Packager

Published on 2018-10-16 by Oleg Nenashev

I would like to introduce Custom WAR Packager - a new tool for Jenkins administrators and developers. This tool allows packaging custom Jenkins distributions as WAR files, Docker images and Jenkinsfile Runner bundles. This tool allows packaging Jenkins, plugins, and configurations in a ready-to-fly distribution. Custom WAR packager is a part of the Ephemeral Jenkins master toolchain which we presented in our A Cloud Native Jenkins blogpost. This toolchain is already used in Jenkins X to package serverless images.

In this blogpost I will show some common use-cases for Custom WAR Packager.

History

As with Jenkins itself, Custom WAR Packager started as a small development tool. For a long time it was a problem to run integration testing in Jenkins. We have 3 main frameworks for it: Jenkins Test Harness, Acceptance Test Harness, and Plugin Compatibility Tester. All these frameworks require a Jenkins WAR file to be passed to them to run tests. What if you want to run Jenkins tests in a custom environment like AWS? Or what if you want to reuse existing Jenkins Pipeline tests and to run them against Pluggable Storage to ensure there are no regressions?

And it was not just an idle question. There were major activities happening in the Jenkins project: Cloud-Native Jenkins, Jenkins Evergreen, and Jenkins X. All these activities required a lot of integration testing to enable Continuous Delivery flows. In order to do this in existing test frameworks, we needed to package a self-configuring WAR file so that it would be possible to run integration tests in existing frameworks. That is why Custom WAR Packager was created in April 2018. Later it got support for packaging Docker images, and in September 2018 it also got support for Jenkinsfile Runner which was created by Kohsuke Kawaguchi and then improved by Nicolas de Loof.

What’s inside?

Custom WAR packager is a tool which is available as CLI Executable, Maven Plugin, or Docker package. This tool takes input definitions and packages them as requested by the user. Everything is managed by a YAML configuration file:

Custom WAR Packager build flow

The tool supports various types of inputs. The list of plugins can be passed via YAML itself, pom.xml, or a BOM file from JEP-309. Custom WAR Packager supports not only released versions, but also builds deployed to the Incremental repository (CD flow for Jenkins core and plugins - JEP-305) and even direct builds by Git or directory path specifications. It allows building packages from any source, without waiting for official releases. The builds are also pretty fast, because the plugin does caching in the local Maven repository by using commit IDs.

Custom WAR packager also supports the following self-configuration options:

WAR Packaging

WAR packaging happens by default every time the repo is built. Generally Custom WAR Packager repackages all inputs into a single WAR file by following conventions defined in the Jenkins core and the JCasC plugin.

Sample configuration:

bundle:
  groupId: "io.jenkins.tools.war-packager.demo"
  artifactId: "blogpost-demo"
  vendor: "Jenkins project"
  description: "Just a demo for the blogpost"
war:
  groupId: "org.jenkins-ci.main"
  artifactId: "jenkins-war"
  source:
    version: 2.138.2
plugins:
  - groupId: "io.jenkins"
    artifactId: "configuration-as-code"
    source:
      # Common release
      version: 1.0-rc2
  - groupId: "io.jenkins"
    artifactId: "artifact-manager-s3"
    source:
      # Incrementals
      version: 1.2-rc259.c9d60bf2f88c
  - groupId: "org.jenkins-ci.plugins.workflow"
    artifactId: "workflow-job"
    source:
      # Git
      git: https://github.com/jglick/workflow-job-plugin.git
      commit: 18d78f305a4526af9cdf3a7b68eb9caf97c7cfbc
  # etc.
systemProperties:
    jenkins.model.Jenkins.slaveAgentPort: "9000"
    jenkins.model.Jenkins.slaveAgentPortEnforce: "true"
groovyHooks:
  - type: "init"
    id: "initScripts"
    source:
      dir: src/main/groovy
casc:
  - id: "jcasc"
    source:
      dir: casc.yml

Docker packaging

In order to do the Docker packaging, Custom WAR Packager uses the official jenkins/jenkins Docker images or other images using the same format. During the build the WAR file just gets replaced by the one built by the tool. It means that ALL image features are available for such custom builds: plugins.txt, Java options, Groovy hooks, etc., etc.

## ...
## WAR configuration from above
## ...

buildSettings:
  docker:
    build: true
    # Base image
    base: "jenkins/jenkins:2.138.2"
    # Tag to set for the produced image
    tag: "jenkins/custom-war-packager-casc-demo"

For example, this demo shows packaging of a Docker image with External Build Logging to Elasticsearch. Although the implementations have been improved as a part of JEP-207 and JEP-210, you can check out this demo to see how the Docker image does self-configuration, connects to a Elasicsearch, and then starts externally storing logs without changes in build log UIs. A Docker Compose file for running the entire cluster is included.

Jenkinsfile Runner packaging

This is probably the most tricky mode of Jenkinsfile Runner. In March a new Jenkinsfile Runner project was announced in the developer mailing list. The main idea is to support running Jenkins Pipeline in a single-shot master mode when the instance just executes a single run and prints outputs to the console. Jenkinsfile Runner runs as CLI or as a Docker image. Custom WAR Packager is able to produce both, though only Docker run mode is recommended. With Jenkinsfile Runner you can run Pipelines simply as…​

docker run --rm -v $PWD/Jenkinsfile:/workspace/Jenkinsfile acmeorg/jenkinsfile-runner

When we started working on Ephemeral (aka "single-shot") masters in the Cloud Native SIG, there was an idea to use Custom WAR Packager and other existing tools (Jenkinsfile Runner, Jenkins Configuration as Code, etc.) to implement it. It would be possible to just replace Jenkins core JAR and add plugins to Jenkinsfile Runner, but it is not enough. To be efficient, Jenkinsfile Runner images should start up FAST, really fast. In the build flow implementation we used some experimental options available in Jenkins and Jenkinsfile Runner, including classloader precaching, plugin unarchiving, etc, etc. With such patches Jenkins starts up in few seconds with configuration-as-code and dozens of bundled plugins.

So, how to build custom Jenkinsfile Runner images? Although there is no release so far, it is not something which can stop us as you see above.

##...
## WAR Configuration from above
##...

buildSettings:
  jenkinsfileRunner:
    source:
      groupId: "io.jenkins"
      artifactId: "jenkinsfile-runner"
      build:
        noCache: true
      source:
        git: https://github.com/jenkinsci/jenkinsfile-runner.git
        commit: 8ff9b1e9a097e629c5fbffca9a3d69750097ecc4
    docker:
      base: "jenkins/jenkins:2.138.2"
      tag: "onenashev/cwp-jenkinsfile-runner-demo"
      build: true

You can find a Demo of Jenkinsfile Runner packaging with Custom WAR Packager here.

More info

There are many other features which are not described in this blogpost. For example, it is possible to alter Maven build settings or to add/replace libraries within the Jenkins core (e.g. Remoting). Please see the Custom WAR Packager documentation for more information. There are a number of demos available in the repository.

If you are interested to contribute to the repository, please create pull requests and CC @oleg-nenashev and Raul Arabaolaza who is the second maintainer now working on Jenkins test automation flows.

What’s next?

There are still many improvements that could be made to the tool to make it more efficient:

  • Add upper bounds checks for transitive plugin dependencies so that the conflicts are discovered during the build

  • Allow passing all kinds of system properties and Java options via configuration YAML

  • Improve Jenkinsfile Runner to improve performance

  • Integrate the tool into Jenkins Integration test flows (see essentialsTest() in the Jenkins Pipeline library)

Many other tasks could be implemented in Custom WAR Packager, but even now it is available to all Jenkins users so that they can build their own Jenkins bundles with it.

Want to know more?

If you are going to DevOps World - Jenkins World in Nice on Oct 22-25, I will be presenting Custom WAR Packager at the Community Booth during the lunch demo sessions. We will be also repeating our A Cloud Native Jenkins talk together with Carlos Sanchez where we will show how Ephemeral Jenkins works with Pluggable Storage. Jenkins X team is also going to present their project using Custom WAR Packager.

Come meet Oleg and other Cloud Native SIG members at DevOps World - Jenkins World on October 22-25 in Nice. register with the code JWFOSS for a 30% discount off your pass.
About the Author
Oleg Nenashev

R&D and Automation engineer with Hardware/Embedded background. He started using Hudson/Jenkins in 2008 and became an active contributor in 2012. Oleg maintains Jenkins core, Remoting and several plugins. He organizes events like Jenkins meetups and GSoC in the community. Nowadays Oleg works at CloudBees on making Jenkins cloud-native.