Buttle Your Code

Jenkins is one of the most popular continuous integration and continuous delivery (CI/CD) tools in the DevOps world. During its lifetime, its configuration process has evolved: A complex GUI wizard for configuration appeared, DSL jobs and pipeline plugins debuted, and more and more Groovy scripts for initializing startup popped up around the Internet. From my point of view, the weakest elements were those initializing scripts, because each company – or even teams in the same company – implemented them differently. Today, thanks to Jenkins Configuration as Code (JCasC), you can fine tune both Jenkins and its plugins with one common approach through the use of easily understandable YAML files.

Jenkins Startup

When Jenkins starts, it reads all command-line parameters and some environmental variables and executes all *.groovy files in lexical order under the ${JENKINS_HOME}/init.groovy.d/ directory; therefore, administrators must learn the Groovy language to prepare scripts. Groovy is a nice, scriptable offspring of Java that lacks its elder's private variables and methods and is used in many Java virtual machine (JVM) systems. In Jenkins, Groovy can be used to modify virtually any aspect of a given CI instance. However, as stated above, you have to be able to write and maintain complex scripts in "yet another language."

Thanks to the JCasC authors, most of the initializing Groovy code becomes obsolete and replaceable by far more readable and maintainable YAML files.

The Chicken and the Egg

JCasC can install plugins, even specific versions, but it is a plugin itself; therefore, it has to be installed on Jenkins. Here comes the chicken and the egg problem: To install plugins, you have to have a plugin that installs them. This obstacle is easily solvable in Docker-based Jenkins instances by using the install-plugins.sh script. Personally, I prefer to install plugins on my own by the simplest method; hence, my Jenkins installation is one Dockerfile (Listing 1) that launches a container with Jenkins and the JCasC plugin. With this code, you can set up Jenkins and install all the components required by JCasC.

01 FROM openjdk:8-jdk-alpine
02
03 RUN apk add --no-cache git openssh-client curl bash # for Jenkins AWT
04    ttf-dejavu
05
06 ARG JENKINS_USER=jenkins
07 ARG UID=1000
08 ARG HTTP_PORT=8080
09 ARG JENKINS_HOME=/ephemeral/jenkins
10
11 # for main web interface:
12 EXPOSE ${HTTP_PORT}
13
14 ENV JENKINS_HOME=${JENKINS_HOME}
15 ENV CASC_JENKINS_CONFIG=/jcasc_config
16
17 COPY jcasc_config/* ${CASC_JENKINS_CONFIG}/
18
19 RUN mkdir -p ${CASC_JENKINS_CONFIG} && mkdir -p ${JENKINS_HOME}/plugins && adduser -h ${JENKINS_HOME} -u ${UID} -s /bin/bash -D ${JENKINS_USER} && chown -R ${UID} ${JENKINS_HOME} ${CASC_JENKINS_CONFIG}
20
21 # Jenkins home directory is a volume, so configuration and build history
22 # can be persisted and survive image upgrades
23 VOLUME ${JENKINS_HOME}
24 USER ${JENKINS_USER}
25
26 ARG JENKINS_UC=https://updates.jenkins.io/stable-2.150/latest/
27 ENV PLUGINS="jdk-tool script-security command-launcher configuration-as-code configuration-as-code-support configuration-as-code-groovy"
28
29 RUN curl -sSfL --connect-timeout 20 --retry 3 --retry-delay 0 --retry-max-time 60 ${JENKINS_UC}/jenkins.war -o ${JENKINS_HOME}/jenkins.war
30 RUN for P in ${PLUGINS}; do curl -sSfL --connect-timeout 20 --retry 2 --retry-delay 0 --retry-max-time 60 ${JENKINS_UC}/${P}.hpi -o ${JENKINS_HOME}/plugins/${P}.jpi; done
31
32 ENV JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Djenkins.install.runSetupWizard=false ${JENKINS_JAVA_OPTIONS:-}"
33 ENTRYPOINT java ${JENKINS_JAVA_OPTIONS} -jar ${JENKINS_HOME}/jenkins.war

First Configuration

Now that JCasC is present, you can make use of it. To begin, assume you need an update center proxy setup, a few additional plugins, LDAP authorization, a few administrators, the Jenkins URL, and a seed job that can create others on demand.

First things first: You can set up one big YAML file that does everything, but I honestly prefer small files, executed in a controlled order. The YAML file can go wherever you want. The CASC_JENKINS_CONFIG environment variable should be set to point to a single YAML file or directory, from which all *.yaml or *.yml files will be executed in alphabetical order. If you do not set the CASC_JENKINS_CONFIG variable, the plugin will look for a single config file in ${JENKINS_ROOT}/jenkins.yaml by default.

For easier maintenance, I put each configuration step in a separate file. JCasC loads files in alphabetical order, so each file is prefixed with two digits to control the execution sequence. To make this solution future-proof, I left gaps between file names in case I have to add other files later that need to be launched at a specific stage.

Setting up the Jenkins host URL though JCasC requires three lines:

### 02_baseURL.yml ###
unclassified:
 location:
   url: "http://misiu.pl:8080/"

Compare this with the code needed in a Groovy script (Listing 2). As you can see, Groovy is noticably more complicated, and the YAML file is more easily read.

01 import jenkins.model.JenkinsLocationConfiguration;
02
03 String newRootURL = "http://misiu.pl:8080/";
04
05 JenkinsLocationConfiguration jlc = JenkinsLocationConfiguration.get()
06
07 jlc.setUrl(newRootURL);
08 jlc.save();

Listing 3 shows two sample YAML files used by JCasC to set up plugins. The first file (05_proxy.yml) sets the proxy for downloading plugins, and the second (10_plugins.yml) defines the proxy server access credentials (discussed later). Once the proxy is ready, you can install plugins.

01 ### 05_proxy.yml ###
02 plugins:
03  proxy:
04    name: "14.3.19.91"
05    port: 8080
06
07 ### 10_plugins.yml ###
08 plugins:
09  required:
10    matrix-auth: latest
11    ldap: latest
12    my-pro-plugin: http://download.mis.com/my-pro-plugin-3.14.91.jpi

The definition of a plugin comprises a name and version – latest or explicit (e.g., 3.14) or the URL to the .hpi source. When installing plugins, you should note that the explicit version does not work with all update centers. Moreover, JCasC does not perform a Jenkins restart, so with a version change, more complex installations might be unstable without a manual restart. The JCasC team and Jenkins project are considering improving this plugin "feature," because it is a bit unreliable.

Access configuration consists of connecting to the LDAP server and assigning the appropriate permissions to chosen users, which you can put all in one YAML script (Listing 4). First, you configure the LDAP server. A YAML reference along with some examples are available in the GitHub plugins repository [3], so you don't need to memorize it here. Second, you define the permission strategy. All options are available, so I chose the most popular one: globalMatrix (line 12). Each permission is defined as a new entry in the list of strings.

01 jenkins:
02  securityRealm:
03    ldap:
04      configurations:
05        - groupMembershipStrategy:
06            fromUserRecord:
07              attributeName: "user"
08          inhibitInferRootDN: false
09          rootDN: "dc=amecme,dc=org"
10          server: "ldaps://ldap.szandala.org"
11  authorizationStrategy:
12    globalMatrix:
13      grantedPermissions:
14        - "Overall/Read:anonymous"
15        - "Overall/Administer:szandala"
16        - "Job/Configure:karolinka"

Keeping Secrets

Everyone has secrets, so sometimes during Jenkins configuration you need to use credentials. JCasC comes with support for HashiCorp Vault, an external tool that stores and controls access to secrets and other sensitive data and allows you to use encrypted values and decrypt them on the fly during configuration. To begin, you need a running instance of HashiCorp Vault and credentials – a token or username and password – to access Vault. Next, export a few variables or prepare a secrets file, set in the CASC_VAULT_FILE environment variable:

CASC_VAULT_PW=PASSWORD
CASC_VAULT_USER=medivh
CASC_VAULT_TOKEN=TOKEN
CASC_VAULT_PATH=secret/jenkins/master
CASC_VAULT_URL=https://vault.szandala.com
CASC_VAULT_MOUNT=ldap

Now you can feed data to Vault:

vault write   -address=https://vault.szandala.com   secret/jenkins/master   SECRET_PASS="garona-halforcen"   SSH_MAGIC_KEY=@/vault/file/secrets/  jenkins_ssh_key

JCasC then substitutes variables in YAML with data from Vault (Listing 5).

01 credentials:
02   system:
03     domainCredentials:
04       - credentials:
05           - usernamePassword:
06               scope: "GLOBAL"
07               id: "szandala"
08               username: "szandala"
09               password: "${SECRET_PASS}"
10               description: "User szandala's credentials"

Create a Job

Configuration is not just about setting up the Jenkins master; it can also include creating an initial set of jobs with job-dsl-plugin and a job-dsl script, which requires the Configuration as Code Support plugin in addition to the basic JCasC plugin already installed; otherwise, the job's root element cannot be used.

The Job DSL plugin uses Groovy syntax for its job configuration DSL (domain-specific language), so you have to mix YAML and Groovy within a YAML file. Listing 6 shows a YAML script with a jobs section that creates a so-called seed job that is used later to deploy other jobs.

01 jenkins:
02   systemMessage: "Creating SEED job"
03 jobs:
04   - script: |
05         freeStyleJob('TOOL_JobsMakel').with {
06             displayName('Deploy Jenkins job definitions')
07             label('master')
08             parameters {
09                 stringParam('SCM_BRANCH','master',
10                     "Source branch for SCM repository, default is master")
11             }
12             scm {
13                 git('${SCM_REPO}', '${SCM_BRANCH}')
14             }
15             steps {
16                 dsl(['jenkins_jobs/*.groovy'])
17             }
18         }

Last Resort

Sadly, YAML configuration will hardly ever cover 100% of a Jenkins configuration. That gap could be easily filled with Groovy initializing scripts, but they come with disadvantages: They require a bit of Groovy knowledge, and they cannot be "reloaded" easily without a Jenkins restart. To overcome these cons, the configuration-as-code-groovy supporting plugin offers a last resort in the form of a groovy element with a list of Groovy scripts to be run. Although this solution still requires a bit of Groovy proficiency, the scripts can be rerun along with a basic JCasC reload.

The best example of an unsupported plugin is gerrit-trigger, which allows Jenkins to listen to events on a Gerrit server; however, before it can start listening, the host URL has to be added to the Jenkins configuration. Basic JCasC cannot do this (at the time of this writing), so the setup can be done through the GUI or a Groovy script. Because the GUI is considered impractical for true DevOps, you need to put the Groovy script into JCasC YAML (Listing 7).

01 groovy:
02  - script: |
03      import jenkins.model.Jenkins;
04      import net.sf.json.JSONObject;
05      import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer;
06
07      if ( Jenkins.instance.pluginManager.activePlugins.find { it.shortName == "gerrit-trigger" } != null ) {
08          println("Setting gerrit-trigger server plugin");
09
10          def gerritPlugin = Jenkins.instance.getPlugin(com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl.class);
11          gerritPlugin.getPluginConfig().setNumberOfReceivingWorkerThreads(3);
12          gerritPlugin.getPluginConfig().setNumberOfSendingWorkerThreads(1);
13
14          def serverName = "grenoble-gerrit";
15          GerritServer server = new GerritServer(serverName);
16          def config = server.getConfig();
17
18          def triggerConfig = [
19              'gerritHostName':"gerrit.com",
20              'gerritSshPort':29418,
21              'gerritUserName':"szandala_jenkins",
22              'gerritFrontEndUrl':"https://gerrit.com"
23          ];
24
25          config.setValues(JSONObject.fromObject(triggerConfig));
26          server.setConfig(config);
27
28          // avoid duplicate servers on the server list;
29          if ( gerritPlugin.containsServer(serverName) ) {
30              gerritPlugin.removeServer(gerritPlugin.getServer(serverName));
31          }
32
33          gerritPlugin.addServer(server);
34          server.start();
35          server.startConnection();
36          println("Setting ${serverName} completed");

As you can see, it is just a copy of the init.groovy.d script as inline code to YAML. For the sake of readability, you can move this snippet to a separate file (e.g., GerritTriggerConfigurator.groovy) and run it locally (with a file predicate) or remotely (e.g., on GitHub; Listing 8).

01 groovy:
02  - file: /tmp/GerritTriggerConfigurator.groovy
03  - url: https://raw.githubusercontent.com/szandala/configuration-as-code-plugin/master/integrations/src/test/resources/io/jenkins/plugins/casc/GroovySetProxy.yml

GUI

JCasC is intended to configure Jenkins without a GUI; however, when Jenkins is already running, you might need an immediate configuration deployment. When you install JCasC, you can find one more entry in the Jenkins configuration (Figure 1). (This view is also available from the https://jenkins.url/configuration-as-code/ URL.)

In this panel you can find (1) the source of the currently loaded configuration, with (2) an option to load other YAMLs. Keep in mind that on a Jenkins restart, if you have exported CASC_JENKINS_CONFIG, it will be used instead of the value inserted here. Two more actions (3) reload an existing configuration, which obviously reloads a setup from a known path, and (4) generate a single YAML file, which should be as close as possible to the configuration for your Jenkins. This setup is far from perfect; as the warning states, it should be "used for inspiration" for your final configuration.

Summary

Setting up Jenkins is a complex process, because both Jenkins and its plugins require tuning and configuration, with dozens of parameters to set within the Manage Jenkins section of the Jenkins user interface. JCasC simplifies the entire configuration with the use of human-friendly, easy-to-read YAML syntax. Without any manual steps, this configuration can be validated and applied to a Jenkins master in a fully reproducible way, without any downtime. With JCasC, setting up a new Jenkins master will become a no-brainer.