Group Swim

Over the years, Docker container software has caused a metaphorical tsunami among software developers and, along with agile principles, improved productivity and DevOps working practices to speed code releases. Docker has had an unprecedented global effect on how applications are built and deployed.

Thanks in part though to an eye-watering rate of evolution in the industry, Docker is no longer the default container software. Some members of the commentariat have unfairly already read the company their last rites. However, in this article, I don't ring the death knell for Docker's fantastic container software, because in my opinion, it is far from receiving its coup de grâce. However, I do look at an alternative to Docker called Podman, its feature set, and some notable differences in its container run time relative to Docker.

Docker Dominance

The first version of Docker [1] was released March 20, 2013. Although containers had already been available on Unix-like systems for a few years, Docker's software allowed small units of code to focus on one specific process to run a single service or perform a single task. By deploying these small code units, software developers improved the portability, consistency, and therefore the predictability of the services and tasks they were running.

As Docker Inc. rode on the crest of their very large wave, containers were invariably the talk of the town. Containers helped shape how architects designed, created, and then deployed their software amid an unremitting level of cloud adoption.

As concepts evolved further, microservices gained consensus as the best way forward, which meant modern software applications were broken down into small, modular, and independently deployable chunks. Soon after, cloud native became the way to discuss complex infrastructure based on microservices deployed in containers with an orchestrator (to help manage multiple containers running at once) in the cloud.

At this point, Google drew on an unparalleled 15 years of production experience running a containerized cluster system called Borg and released the open source Kubernetes orchestrator. On an exponentially growing proliferation of popularity, Kubernetes, akin to Docker's stratospheric rise, soon became the orchestrator of choice and in turn began forging the way that container software was to be used.

Now the Docker container software, after pioneering strides, was no longer the default container software used within the orchestrator, which led to Docker focusing more on the professional services side of its business.

In this article for the purposes of simplicity, I'll assume a container run time is the container software required for actually running containers directly. (See the "Oh, I See … OCI" box.)

Rougher and Tougher

The Podman website [5] greets you with a logo of marine mammals, representing the Kubernetes "pod" (one or more containers), and, hence, the run time's name.

At first glance, Podman is intriguing because it is daemonless; that is, it has no background service and the run time is executed only when requested. From a security perspective, that's good news: Podman offers less attack surface, because it's not running around the clock.

Additionally, Podman is OCI-friendly, helping to propagate the results of hard-fought standards battles in an effort to normalize what containers look like and how they act.

To my mind, however, the most valuable addition to Podman's credentials relates to something else. Unlike Docker, which has made great leaps in the right direction but historically struggled to keep all users happy with security, Podman lets users fire up the run time without being the root superuser. When Podman is run as a less privileged user, it sets up and then enters a user namespace. Once inside, Podman runs as a privileged user (but only with privileges inside that namespace) and has permission to mount certain filesystems and create a container. From a DevSecOps perspective, this means developers can run containers without having superuser permissions.

Believe me when I say that this provides a far higher degree of comfort when it comes to leaving containers running on a system – development and production environments alike – 24 hours a day. By design, Linux has a number of sophisticated mechanisms to help isolate critical functionality and keep it away from less privileged users to ensure that only the root user can affect such services.

The container image format also supports other valuable features: image verification, so you know you're using a trusted container image; container image layer management functionality; pod management (managing groups of containers); and resource isolation between containers.

For the non-Linux aficionados among us, a number of other features are in the planning, such as connecting into remote Mac and Windows systems.

Bigger and Bolder

I'm using the Debian derivative Linux Mint, which harks from Ubuntu 18.04. Detailed Podman installation instructions can be found on GitHub [6]. The instructions offer a number of options, as shown in Figure 1, which details the packages required to build Podman from scratch.

For my install, I take the path of least resistance and use the Personal Package Archive (PPA) as root or with sudo:

$ apt-get update -qq
$ apt-get install -qq -y software-properties-common uidmap
$ add-apt-repository -y ppa:projectatomic/ppa
$ apt-get update -qq
$ apt-get -qq -y install podman

You're helpfully advised to check the build and run dependencies on the install page [7] if you encounter any issues.

If you're concerned about what those commands are doing to your machine, then simply remove the -y (automatic "yes" to prompts) and -qq (logging output) options for more details. To check that all is well following those commands, run:

$ podman --help

If you see lots of output and the minor complaint

WARN[0000] unable to find /etc/containers/registries.conf

don't worry, because I'll fix that problem next.

Peas in a Pod

Podman checks a specific set of container image registries defined in the registries.conf file. I used the example configuration file from the installation help page [6] (Listing 1). As you can see in line 12, the following container image registries are to be searched:

• docker.io
• registry.fedoraproject.org
• quay.io
• registry.access.redhat.com
• registry.centos.org

01 # This is a system-wide configuration file used to
02 # keep track of registries for various container backends.
03 # It adheres to TOML format and does not support recursive
04 # lists of registries.
05
06 # The default location for this configuration file is /etc/containers/registries.conf.
07
08 # The only valid categories are: 'registries.search', 'registries.insecure',
09 # and 'registries.block'.
10
11 [registries.search]
12 registries = ['docker.io', 'registry.fedoraproject.org', 'quay.io', 'registry.access.redhat.com', 'registry.centos.org']
13
14 # If you need to access insecure registries, add the registry's fully-qualified name.
15 # An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
16 [registries.insecure]
17 registries = []
18
19 # If you need to block pull access from a registry, uncomment the section
20 # below and add the registries fully-qualified name.
21 #
22 [registries.block]
23 registries = []

These registries are called when Podman can't find an image locally or if an image doesn't contain a fully qualified registry name within an image name itself.

Vanilla Pod

Operationally, Podman makes use of libpod to fulfill your container needs and boldly makes no bones about usurping Docker as the run time, inviting you to add the following line to your .bash_aliases (or similar) file so that you don't have to run it each time you open up a new shell or log in to a machine:

$ alias docker=podman

Alternatively, you can just run it at the command line. Once you've done that, you can test Podman. For example, to pull the popular nginx container image, enter:

$ podman pull nginx

Figure 2 shows what happens when the command is run as root. In many cases, the command syntax is similar to Docker's, for an easy transition. To check which containers are running on Podman, you can use a command that looks like a Docker command:

$ podman ps

To check out the run-time version and technical innards in a bit more detail, you can use the handy command:

$ podman info

If you look carefully at its output, you'll see which registries your configuration file is pointing at, too.

Now, to run the Nginx container, enter:

$ podman run -dit nginx

In Listing 2, you can see the resulting hash, which appears to indicate that the container is running correctly, although with an error. I'll come back to the error in a moment, but first, I'll check that the container is running as hoped by repeating the podman ps command.

$ podman run -dit nginx
ERRO[0000] could not find slirp4netns, the network namespace won't be configured: exec: "slirp4netns": executable file not found in $PATH
0a2091b63bc5de710238fadc68ba3f5e0f9af8800ec7f76fd52a84c49a1ab0a7

Listing 3 shows that I do have a working container, so I'll deal with the network namespace error now.

CONTAINER ID  IMAGE                           COMMAND               CREATED        STATUS
ba99b8a162e2  docker.io/library/nginx:latest  nginx -g daemon o...  4 seconds ago  Up 3 seconds ago

After another look into namespaces on the installation page, a couple of commands jumped out. To fix the problem, you could try running an echo command as root that allows less privileged users to set up namespaces:

$ echo 'kernel.unprivileged_userns_clone=1' > /etc/sysctl.d/userns.conf

However, I suspect the PPA and package installation routes already enabled that functionality, and therefore, it's not the issue in this case.

Monkey Pod

Focusing on the error more closely, the network namespace error indicates that Podman needs network plugins to function correctly. To get networking functionality working, you need to look at the Common Networking Interface (CNI) plugins. The following commands also require you to be the root (or use sudo).

First, download the networking configuration and create a directory for it, if it does not already exist:

$ mkdir -p /etc/cni/net.d
$ curl -qsSL https://raw.githubusercontent.com/containers/libpod/master/cni/87-podman-bridge.conflist | sudo tee /etc/cni/net.d/99-loopback.conf

If you list the files in directory /etc/cni/net.d, you'll see two files:

$ ls /etc/cni/net.d
87-podman-bridge.conflist 99-loopback.conf

Next, install the Go programming language (about 220MB):

$ apt install golang-go

That should provide a sane Go environment for the CNI plugins:

$ git clone https://github.com/containernetworking/plugins.git $GOPATH/src/github.com/containernetworking/plugins

If you have a peek inside the directory now, you should see a build_linux.sh script. Check that it's there, change to its directory, and run the script to build the network plugins:

$ ls $GOPATH/src/github.com/containernetworking/plugins
$ cd $GOPATH/src/github.com/containernetworking/plugins
$ ./build_linux.sh

Once that's completed successfully, you can move the plugins out of the build directory into the cni directory with the following commands:

$ mkdir -p /usr/libexec/cni
$ cp bin/* /usr/libexec/cni

Escape Pod

Before starting up a new container on a Debian derivative, you have to do one more thing, I suspect, because Podman's development appears to be heavily oriented toward the Fedora and Red Hat Enterprise Linux operating systems. On systems that are members of the Debian family, like Ubuntu and Mint, in my case, I need to install a slirp4netns package, which according to Apt, allows "User-mode networking for unprivileged network namespaces."

If you think back to one of the attractive security features – being able to run Podman without being the root user within its own kernel namespace (to isolate it from other containers and the system itself) – this statement makes sense.

To install the package, enter:

$ apt install slirp4netns

To prove that the networking plugins are also working for a non-root user, you should become a less privileged user:

$ su - johncooper

(Replace johncooper with your login name or another username.) Before trying to launch a new container, check with the podman ps command again that the containers are running.

To test the networking, launch another container from the Nginx image:

$ podman run -dit nginx
c73445cacc1f3fc57979c35e62e30bbb9edeff  3712b7841e529ca95a23662dd1

Note that the error is gone now, and a hash is visible. Next, get the hash of your newly launched container with podman ps again. From my Nginx hash above, I use the command

$ podman inspect c734

and scroll through its output to see whether the Nginx web server is running. I see the line:

"IPAddress": "10.88.0.3",

Lo and behold, when I run a curl command to query TCP port 80 on that container's IP address, I get the output seen in Listing 4. Just as hoped, it shows the HTML output from the container, so I can see that all is well.

01 $ curl http://10.88.0.3:80
02
03 <!DOCTYPE html>
04 <html>
05 <head>
06 <title>Welcome to nginx!</title>
07 <style>
08     body {
09         width: 35em;
10         margin: 0 auto;
11         font-family: Tahoma, Verdana, Arial, sans-serif;
12     }
13 </style>
14 </head>
15 <body>
16 <h1>Welcome to nginx!</h1>
17 <p>If you see this page, the nginx web server is successfully installed and
18 working. Further configuration is required.</p>
19
20 <p>For online documentation and support please refer to
21 <a href="http://nginx.org/">nginx.org</a>.<br/>
22 Commercial support is available at
23 <a href="http://nginx.com/">nginx.com</a>.</p>
24
25 <p><em>Thank you for using nginx.</em></p>
26 </body>
27 </html>

The result is an Nginx web server instance running securely as a non-root user, isolated within its own user namespace.

The End Is Nigh

As I suggested at the beginning of this article, a number of intriguing run times are available for you to try. In the fast-moving container space, there's no guarantee which one will come out on top, but CRI-O has won the race in Kubernetes for the time being.

I hope with some experimentation you will want to try running containers as a non-root user and look at how to secure your run time's attack surface further inside user namespaces.

If you are keen to explore more, then check out the clever Buildah [8] package, described as a "a tool that facilitates building OCI container images." If you're familiar with Dockerfiles, used to create Docker container images, you might be interested in the Buildah docs [9].

In the meantime, experiment with Podman and keep a close eye on the container technology space to stay up to date. It moves quickly!