Blueprint

Gone are the days when you spend nights preparing your infrastructure for new software releases. Development cycles are getting shorter and shorter, and development teams are becoming more agile. Another concern is automation, which you can implement with configuration management. For example, just describe the configuration of virtual machines (VMs) to deliver and update them later according to a blueprint.

Terraform [1] by HashiCorp uses this idea to provision or adapt infrastructures. They also provide Vagrant, which handles the deployment of development environments. Terraform not only supports clean provisioning of the infrastructure, it also lets you change already provisioned environments. Terraform expects text configuration files, which are simply known as "configurations," with a .tf file extension. They can be versioned with tools like Git or SVN.

Choose between HashiCorp Configuration Language (HCL) and JSON formats. Although HCL is based on JSON, it supports comments and some other extensions to simplify coding. The code examples in this article use the HCL format.

Blueprint

Terraform does almost all the work involved in deploying the virtual infrastructure, starting with provisioning VMs with VMware and OpenStack in your data center, as well as with Amazon and Oracle cloud providers, extending all the way to adjustments of the DNS or monitoring server. For this purpose, Terraform uses different providers [2] to provide resources for the corresponding platforms, which in turn feed into the configurations.

In this article, I use DigitalOcean [3] to provide insight into how Terraform works and, using the example, to show that the Infrastructure as Code paradigm does not have to be such a big hurdle.

Basics

First, you need accounts for DigitalOcean and Cloudflare, including valid API keys. You also have to configure your own domain with Cloudflare.

Installing Terraform is a very simple procedure. Once you have downloaded the appropriate ZIP archive [4], all you have to do is unpack the binary in the archive and copy it to a location in the filesystem that the $PATH variable covers. For example, you can use /usr/local/bin and then simply run the terraform command.

The goal of the exercise is to place a droplet that is nothing more than a VM with the DigitalOcean cloud provider – in this case, with a preinstalled application, a Docker daemon. A container that provides a web service then starts on the VM. After successful provisioning with DigitalOcean, Terraform should create a suitable DNS entry so that third parties can access the web application automatically by domain and not just IP address.

The plan first requires a folder in which all further configurations are stored. The most important file is variables.tf (Listing 1), which declares all variables used. These can be default values, types, or a description of the variable value.

01 variable "site_name" {
02   description = "Description"
03   type = "string"
04   default = "DEMO-SITE"
05 }
06 variable "site_author" {
07   type = "string"
08   default = "Jon Doe"
09 }
10 variable "site_container" {}
11 variable "do_token" {
12   type = "string"
13 }
14 variable "key_path" {}
15 variable "ssh_priv_key" {}
16 variable "ssh_pub_key" {}
17 variable "cloudflare_email" {}
18 variable "cloudflare_token" {}
19 variable "cloudflare_domain" {}

The next file sets declared variables. It not only depends on the file extension, but also on the file name: terraform.tfvars (Listing 2). Among other things, it contains the API keys for DigitalOcean and Cloudflare. Information on where the you can find such API tokens is provided online [5] [6].

01 do_token = "01189998819991197253"
02 ssh_priv_key = "/home/jon.doe/.ssh/id_rsa"
03 ssh_pub_key = "/home/jon.doe/.ssh/id_rsa.pub"
04 cloudflare_email = "<jon.doe@example.com>"
05 cloudflare_token = "<01189998819991197253>"
06 cloudflare_domain = "<example.com>"
07 site_name = "<mysite.example.com>"
08 site_author = "John Doe"
09 site_container = "dockersamples/static-site"

Resource Planning

Now create one file for the DigitalOcean and Cloudflare configurations. Distributing this information across two files is not necessary, nor is it a convention, but it makes it easier for teams to keep track of versioning and work on the infrastructure.

Terraform automatically reads all files with the .tf* extension. From the digitalocean.tf file (Listing 3), it first loads the digitalocean provider and then transfers the API token from the do_token variable in the terraform.tfvars file. Terraform accesses variables with the "${var.<Variablenname>}" string. In the next step, Terraform calls the digitalocean_ssh_key resource with the jondoe name (line 4), reads the public key from a file on the local filesystem, and uploads it to DigitalOcean. The VM needs it later.

01 provider "digitalocean" {
02     token = "${var.do_token}"
03 }
04 resource "digitalocean_ssh_key" "jondoe" {
05   name       = "Jons Key"
06   public_key = "${file("${var.ssh_pub_key}")}"
07 }
08 resource "digitalocean_droplet" "mywebapp" {
09   image = "docker-16-04"
10   name: guest
11   region = "fra1"
12   size = "512mb"
13   ssh_keys = ["${digitalocean_ssh_key.jondoe.id}"]
14   provisioner "remote-exec" {
15     inline = [
16     "docker run -p 80:80 --name ${var.site_name}          -e AUTHOR=\"${var.site_author}\"          -d -P ${var.site_container}",
17     ]
18     connection {
19       type = "ssh"
20       user = "root"
21       private_key = "${file("${var.ssh_priv_key}")}"
22     }
23   }
24 }
25 output "IP" {
26   value =        "${digitalocean_droplet.mywebapp.ipv4_address}"
27 }

The second digitalocean_droplet resource named mywebapp (line 8) creates a droplet from the docker-16-04 image containing Ubuntu 16.04 and a preinstalled Docker daemon. Thanks to line 13, Terraform accesses an id value of the digitalocean_ssh_keys resource and the jondoe name. The id attribute stores Terraform after successfully perfoming the digitalocean_ssh_keys resource and provides it to all other resources.

At the same time, the file defines a dependency: Terraform only executes digitalocean_droplet if it knows the digitalocean_ssh_key.jondoe.id value. When it accesses resource attributes, Terraform automatically establishes these dependencies and calls the resources accordingly in the correct order. The available attributes can be found in the resource documentation [7]. If Terraform does not automatically recognize other dependencies because they do not use attributes of other resources, you must set them explicitly [8].

From line 25 onward, Listing 3 finally defines an output variable and declares it with an attribute. This variable (IP here), explicitly outputs Terraform on the console when the program has completed its task successfully.

Accessibility

Admins can deploy the Provisioner [9] on resources. Terraform either runs this locally or – with the corresponding connection data – remotely by SSH or WinRM. For example, if you have created a VM, you can store the public IP in a local file on your own workstation or, as in this example, pass remote-exec to the Provisioner, which then runs a command on the droplet you created, thus launching a Docker container.

Before Terraform creates the Cloudflare configuration, you need to test the configuration up to this point. To do this, you first need to start an initialization process with terraform init (Figure 1) that searches all .tf files for providers, checks to see whether the corresponding provider definitions already exist, and downloads them if necessary. You always need to run the command when new providers are used in Terraform configurations.

The command to check the extent to which the infrastructure described in the previous configurations has already been provisioned is

terraform plan -out digitalocean-plan

as shown in Listing 4. Because no infrastructure is in place as yet, Terraform generates all configured resources.

01 $ terraform plan -out digitalocean-plan
02 Refreshing Terraform state in-memory before plan...
03 [...]
04 An execution plan has been generated and is shown below.
05 Resource actions are indicated with the following symbols:
06   + create
07 Terraform will perform the following actions:
08   + digitalocean_droplet.mywebapp
09       id:                   <computed>
10       image:                "docker-16-04"
11 [...]
12   + digitalocean_ssh_key.jondoe
13       id:                   <computed>
14       name:                 "Jon's key"
15 [...]
16 Plan: 2 to add, 0 to change, 0 to destroy.

The -out parameter generates a plan that guarantees that Terraform does exactly what the output is currently displaying. This is especially important in production environments, because something can change between the input of the terraform plan command and the actual implementation of the infrastructure or configuration. For example, Terraform ensures you approve a certain plan as part of a change process and that the software really only implements the changes stored there.

A simple

terraform apply digitalocean-plan

(Figure 2) then tells Terraform to implement the configuration recorded in the digitalocean-plan plan. The step creates or implements the components described above. The last line shows the previously configured IP address of the droplet (not shown) and checks in the browser whether the outside world can reach the web server in the container.

Once the Docker droplet has been created, Terraform also creates a new subdomain for the web application on Cloudflare and adds to it the IP address of the droplet. Terraform needs the cloudflare.tf file (Listing 5) for this, which loads the Cloudflare provider. You can then use the cloudflare_record resource (line 6), which then implements the changes for Cloudflare on the DNS server. Here again, Terraform wants to access a variable that does not exist until after the software has generated the droplet at DigitalOcean.

01 provider "cloudflare" {
02   email = "${var.cloudflare_email}"
03   token = "${var.cloudflare_token}"
04 }
05
06 resource "cloudflare_record" "mywebapp" {
07   domain = "${var.cloudflare_domain}"
08   name   = "terraform"
12   proxied = true
11   ttl    = 120
10   type   = "A"
09   value  = "${digitalocean_droplet.mywebapp.ipv4_address}"
13   provisioner "local-exec" {
14     command = "firefox ${cloudflare_record.mywebapp.hostname}"
15   }
16 }
17
18 output "Website:" {
19   value = "${cloudflare_record.mywebapp.hostname}"
20 }

To create a DNS entry, Terraform needs the public IP address that is in the digitalocean_droplet.mywebapp.ipv4_address attribute of the droplet. Once the cloudflare_record is created, the local-exec provisioner launches Firefox and loads a page with the configured URL.

Thanks to this configuration, Terraform should display the IP addresses and domain names of the websites from which the users access the container. The name is provided by the value in cloudflare_record.mywebapp.hostname.

Make sure Terraform finds all the dependencies for the new cloudflare provider (Listing 5, line 1) by running terraform init. To continue with the deployment,

terraform plan -out cloudflare-plan

creates a new plan describing what Terraform would do (Figure 3). As it turns out, it would only provide the Cloudflare resource here. The call does not change the other resources because they have already been provisioned.

The command in Figure 4,

terraform apply cloudflare-plan

adapts or extends the infrastructure by adding the Cloudflare component. Users can subsequently reach the website at the configured address, which is also in the command output. A quick check in the browser of the URL specified in terraform.tfvars is sufficient to test whether the website exists.

Reset Application

To avoid the need to delete every single component manually, you can remove the provisioned infrastructure with:

terraform destroy

However, nothing happens without explicit confirmation by typing yes in the dialog, which is a good thing, because you will not want the infrastructure to disappear as the result of an accidental destroy command without due consideration.

Conclusions

The example shows how Terraform succeeds in providing infrastructure and expanding it dynamically. The plan subcommand displays planned changes beforehand, allowing for a review. If you delete a DNS entry (e.g., Cloudflare's), Terraform identifies this change and corrects it with apply. This confirmation does not affect other resources. Because many providers already exist, it is quite easy to enter the world of Infrastructure as Code, thanks to Terraform.