
Jenkins Configuration as Code
Buttle Your Code
Jenkins is one of the most popular continuous integration and continuous delivery (CI/CD) tools in the DevOps world. During its lifetime, its configuration process has evolved: A complex GUI wizard for configuration appeared, DSL jobs and pipeline plugins debuted, and more and more Groovy scripts for initializing startup popped up around the Internet. From my point of view, the weakest elements were those initializing scripts, because each company – or even teams in the same company – implemented them differently. Today, thanks to Jenkins Configuration as Code (JCasC), you can fine tune both Jenkins and its plugins with one common approach through the use of easily understandable YAML files.
Jenkins Startup
When Jenkins starts, it reads all command-line parameters and some environmental variables and executes all *.groovy
files in lexical order under the ${JENKINS_HOME}/init.groovy.d/
directory; therefore, administrators must learn the Groovy language to prepare scripts. Groovy is a nice, scriptable offspring of Java that lacks its elder's private variables and methods and is used in many Java virtual machine (JVM) systems. In Jenkins, Groovy can be used to modify virtually any aspect of a given CI instance. However, as stated above, you have to be able to write and maintain complex scripts in "yet another language."
Thanks to the JCasC authors, most of the initializing Groovy code becomes obsolete and replaceable by far more readable and maintainable YAML files.
![Jenkins project logo [1] CC BY-SA 3.0 [2] Jenkins project logo [1] CC BY-SA 3.0 [2]](images/logo.png)
Jenkins project logo [1] CC BY-SA 3.0 [2]
The Chicken and the Egg
JCasC can install plugins, even specific versions, but it is a plugin itself; therefore, it has to be installed on Jenkins. Here comes the chicken and the egg problem: To install plugins, you have to have a plugin that installs them. This obstacle is easily solvable in Docker-based Jenkins instances by using the install-plugins.sh
script. Personally, I prefer to install plugins on my own by the simplest method; hence, my Jenkins installation is one Dockerfile (Listing 1) that launches a container with Jenkins and the JCasC plugin. With this code, you can set up Jenkins and install all the components required by JCasC.
Listing 1: Sample Dockerfile
01 FROM openjdk:8-jdk-alpine 02 03 RUN apk add --no-cache git openssh-client curl bash # for Jenkins AWT 04 ttf-dejavu 05 06 ARG JENKINS_USER=jenkins 07 ARG UID=1000 08 ARG HTTP_PORT=8080 09 ARG JENKINS_HOME=/ephemeral/jenkins 10 11 # for main web interface: 12 EXPOSE ${HTTP_PORT} 13 14 ENV JENKINS_HOME=${JENKINS_HOME} 15 ENV CASC_JENKINS_CONFIG=/jcasc_config 16 17 COPY jcasc_config/* ${CASC_JENKINS_CONFIG}/ 18 19 RUN mkdir -p ${CASC_JENKINS_CONFIG} && mkdir -p ${JENKINS_HOME}/plugins && adduser -h ${JENKINS_HOME} -u ${UID} -s /bin/bash -D ${JENKINS_USER} && chown -R ${UID} ${JENKINS_HOME} ${CASC_JENKINS_CONFIG} 20 21 # Jenkins home directory is a volume, so configuration and build history 22 # can be persisted and survive image upgrades 23 VOLUME ${JENKINS_HOME} 24 USER ${JENKINS_USER} 25 26 ARG JENKINS_UC=https://updates.jenkins.io/stable-2.150/latest/ 27 ENV PLUGINS="jdk-tool script-security command-launcher configuration-as-code configuration-as-code-support configuration-as-code-groovy" 28 29 RUN curl -sSfL --connect-timeout 20 --retry 3 --retry-delay 0 --retry-max-time 60 ${JENKINS_UC}/jenkins.war -o ${JENKINS_HOME}/jenkins.war 30 RUN for P in ${PLUGINS}; do curl -sSfL --connect-timeout 20 --retry 2 --retry-delay 0 --retry-max-time 60 ${JENKINS_UC}/${P}.hpi -o ${JENKINS_HOME}/plugins/${P}.jpi; done 31 32 ENV JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Djenkins.install.runSetupWizard=false ${JENKINS_JAVA_OPTIONS:-}" 33 ENTRYPOINT java ${JENKINS_JAVA_OPTIONS} -jar ${JENKINS_HOME}/jenkins.war
First Configuration
Now that JCasC is present, you can make use of it. To begin, assume you need an update center proxy setup, a few additional plugins, LDAP authorization, a few administrators, the Jenkins URL, and a seed job that can create others on demand.
First things first: You can set up one big YAML file that does everything, but I honestly prefer small files, executed in a controlled order. The YAML file can go wherever you want. The CASC_JENKINS_CONFIG
environment variable should be set to point to a single YAML file or directory, from which all *.yaml
or *.yml
files will be executed in alphabetical order. If you do not set the CASC_JENKINS_CONFIG
variable, the plugin will look for a single config file in ${JENKINS_ROOT}/jenkins.yaml
by default.
For easier maintenance, I put each configuration step in a separate file. JCasC loads files in alphabetical order, so each file is prefixed with two digits to control the execution sequence. To make this solution future-proof, I left gaps between file names in case I have to add other files later that need to be launched at a specific stage.
Setting up the Jenkins host URL though JCasC requires three lines:
### 02_baseURL.yml ### unclassified: location: url: "http://misiu.pl:8080/"
Compare this with the code needed in a Groovy script (Listing 2). As you can see, Groovy is noticably more complicated, and the YAML file is more easily read.
Listing 2: Groovy Config Example
01 import jenkins.model.JenkinsLocationConfiguration; 02 03 String newRootURL = "http://misiu.pl:8080/"; 04 05 JenkinsLocationConfiguration jlc = JenkinsLocationConfiguration.get() 06 07 jlc.setUrl(newRootURL); 08 jlc.save();
Listing 3 shows two sample YAML files used by JCasC to set up plugins. The first file (05_proxy.yml
) sets the proxy for downloading plugins, and the second (10_plugins.yml
) defines the proxy server access credentials (discussed later). Once the proxy is ready, you can install plugins.
Listing 3: Setting Up Plugins
01 ### 05_proxy.yml ### 02 plugins: 03 proxy: 04 name: "14.3.19.91" 05 port: 8080 06 07 ### 10_plugins.yml ### 08 plugins: 09 required: 10 matrix-auth: latest 11 ldap: latest 12 my-pro-plugin: http://download.mis.com/my-pro-plugin-3.14.91.jpi
The definition of a plugin comprises a name and version – latest or explicit (e.g., 3.14) or the URL to the .hpi
source. When installing plugins, you should note that the explicit version does not work with all update centers. Moreover, JCasC does not perform a Jenkins restart, so with a version change, more complex installations might be unstable without a manual restart. The JCasC team and Jenkins project are considering improving this plugin "feature," because it is a bit unreliable.
Access configuration consists of connecting to the LDAP server and assigning the appropriate permissions to chosen users, which you can put all in one YAML script (Listing 4). First, you configure the LDAP server. A YAML reference along with some examples are available in the GitHub plugins repository [3], so you don't need to memorize it here. Second, you define the permission strategy. All options are available, so I chose the most popular one: globalMatrix
(line 12). Each permission is defined as a new entry in the list of strings.
Listing 4: Configuring Matrix Authorization
01 jenkins: 02 securityRealm: 03 ldap: 04 configurations: 05 - groupMembershipStrategy: 06 fromUserRecord: 07 attributeName: "user" 08 inhibitInferRootDN: false 09 rootDN: "dc=amecme,dc=org" 10 server: "ldaps://ldap.szandala.org" 11 authorizationStrategy: 12 globalMatrix: 13 grantedPermissions: 14 - "Overall/Read:anonymous" 15 - "Overall/Administer:szandala" 16 - "Job/Configure:karolinka"
Keeping Secrets
Everyone has secrets, so sometimes during Jenkins configuration you need to use credentials. JCasC comes with support for HashiCorp Vault, an external tool that stores and controls access to secrets and other sensitive data and allows you to use encrypted values and decrypt them on the fly during configuration. To begin, you need a running instance of HashiCorp Vault and credentials – a token or username and password – to access Vault. Next, export a few variables or prepare a secrets file, set in the CASC_VAULT_FILE
environment variable:
CASC_VAULT_PW=PASSWORD CASC_VAULT_USER=medivh CASC_VAULT_TOKEN=TOKEN CASC_VAULT_PATH=secret/jenkins/master CASC_VAULT_URL=https://vault.szandala.com CASC_VAULT_MOUNT=ldap
Now you can feed data to Vault:
vault write -address=https://vault.szandala.com secret/jenkins/master SECRET_PASS="garona-halforcen" SSH_MAGIC_KEY=@/vault/file/secrets/ jenkins_ssh_key
JCasC then substitutes variables in YAML with data from Vault (Listing 5).
Listing 5: JCasC YAML
01 credentials: 02 system: 03 domainCredentials: 04 - credentials: 05 - usernamePassword: 06 scope: "GLOBAL" 07 id: "szandala" 08 username: "szandala" 09 password: "${SECRET_PASS}" 10 description: "User szandala's credentials"
Create a Job
Configuration is not just about setting up the Jenkins master; it can also include creating an initial set of jobs with job-dsl-plugin and a job-dsl
script, which requires the Configuration as Code Support plugin in addition to the basic JCasC plugin already installed; otherwise, the job's root element cannot be used.
The Job DSL plugin uses Groovy syntax for its job configuration DSL (domain-specific language), so you have to mix YAML and Groovy within a YAML file. Listing 6 shows a YAML script with a jobs
section that creates a so-called seed job that is used later to deploy other jobs.
Listing 6: YAML-Groovy Hybrid File
01 jenkins: 02 systemMessage: "Creating SEED job" 03 jobs: 04 - script: | 05 freeStyleJob('TOOL_JobsMakel').with { 06 displayName('Deploy Jenkins job definitions') 07 label('master') 08 parameters { 09 stringParam('SCM_BRANCH','master', 10 "Source branch for SCM repository, default is master") 11 } 12 scm { 13 git('${SCM_REPO}', '${SCM_BRANCH}') 14 } 15 steps { 16 dsl(['jenkins_jobs/*.groovy']) 17 } 18 }
Last Resort
Sadly, YAML configuration will hardly ever cover 100% of a Jenkins configuration. That gap could be easily filled with Groovy initializing scripts, but they come with disadvantages: They require a bit of Groovy knowledge, and they cannot be "reloaded" easily without a Jenkins restart. To overcome these cons, the configuration-as-code-groovy supporting plugin offers a last resort in the form of a groovy
element with a list of Groovy scripts to be run. Although this solution still requires a bit of Groovy proficiency, the scripts can be rerun along with a basic JCasC reload.
The best example of an unsupported plugin is gerrit-trigger, which allows Jenkins to listen to events on a Gerrit server; however, before it can start listening, the host URL has to be added to the Jenkins configuration. Basic JCasC cannot do this (at the time of this writing), so the setup can be done through the GUI or a Groovy script. Because the GUI is considered impractical for true DevOps, you need to put the Groovy script into JCasC YAML (Listing 7).
Listing 7: YAML groovy Section
01 groovy: 02 - script: | 03 import jenkins.model.Jenkins; 04 import net.sf.json.JSONObject; 05 import com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServer; 06 07 if ( Jenkins.instance.pluginManager.activePlugins.find { it.shortName == "gerrit-trigger" } != null ) { 08 println("Setting gerrit-trigger server plugin"); 09 10 def gerritPlugin = Jenkins.instance.getPlugin(com.sonyericsson.hudson.plugins.gerrit.trigger.PluginImpl.class); 11 gerritPlugin.getPluginConfig().setNumberOfReceivingWorkerThreads(3); 12 gerritPlugin.getPluginConfig().setNumberOfSendingWorkerThreads(1); 13 14 def serverName = "grenoble-gerrit"; 15 GerritServer server = new GerritServer(serverName); 16 def config = server.getConfig(); 17 18 def triggerConfig = [ 19 'gerritHostName':"gerrit.com", 20 'gerritSshPort':29418, 21 'gerritUserName':"szandala_jenkins", 22 'gerritFrontEndUrl':"https://gerrit.com" 23 ]; 24 25 config.setValues(JSONObject.fromObject(triggerConfig)); 26 server.setConfig(config); 27 28 // avoid duplicate servers on the server list; 29 if ( gerritPlugin.containsServer(serverName) ) { 30 gerritPlugin.removeServer(gerritPlugin.getServer(serverName)); 31 } 32 33 gerritPlugin.addServer(server); 34 server.start(); 35 server.startConnection(); 36 println("Setting ${serverName} completed");
As you can see, it is just a copy of the init.groovy.d
script as inline code to YAML. For the sake of readability, you can move this snippet to a separate file (e.g., GerritTriggerConfigurator.groovy
) and run it locally (with a file predicate) or remotely (e.g., on GitHub; Listing 8).
Listing 8: Local/Remote Runs
01 groovy: 02 - file: /tmp/GerritTriggerConfigurator.groovy 03 - url: https://raw.githubusercontent.com/szandala/configuration-as-code-plugin/master/integrations/src/test/resources/io/jenkins/plugins/casc/GroovySetProxy.yml
GUI
JCasC is intended to configure Jenkins without a GUI; however, when Jenkins is already running, you might need an immediate configuration deployment. When you install JCasC, you can find one more entry in the Jenkins configuration (Figure 1). (This view is also available from the https://jenkins.url/configuration-as-code/ URL.)

In this panel you can find (1) the source of the currently loaded configuration, with (2) an option to load other YAMLs. Keep in mind that on a Jenkins restart, if you have exported CASC_JENKINS_CONFIG
, it will be used instead of the value inserted here. Two more actions (3) reload an existing configuration, which obviously reloads a setup from a known path, and (4) generate a single YAML file, which should be as close as possible to the configuration for your Jenkins. This setup is far from perfect; as the warning states, it should be "used for inspiration" for your final configuration.
Summary
Setting up Jenkins is a complex process, because both Jenkins and its plugins require tuning and configuration, with dozens of parameters to set within the Manage Jenkins section of the Jenkins user interface. JCasC simplifies the entire configuration with the use of human-friendly, easy-to-read YAML syntax. Without any manual steps, this configuration can be validated and applied to a Jenkins master in a fully reproducible way, without any downtime. With JCasC, setting up a new Jenkins master will become a no-brainer.