Batten the Hatches

Whether you compile Apache yourself or use a package from a repository, you need to keep the software up to date and shut down vulnerabilities as soon as possible. One way to keep on top of important information is to subscribe to Apache's Announce mailing list [1].

In addition to the web server software itself, you also need to ensure that interpreters such as PHP, Python, and Perl and the web applications you use are secured. Last but not least, every security-conscious admin patches the underlying operating system on an ongoing basis.

Installation, Modules, and Updates

As a first step, you should disable unnecessary modules. On Debian and Ubuntu this task is quite easy thanks to the a2dismod command. Otherwise, you will have to search for the LoadModule directive.

The primary candidates for disabling include autoindex, CGI/CGId, Include, UserDir, and suEXEC. To determine which modules are already built-in, type apache2 -l, and after disabling modules, call the apache2ctl -t command before restarting Apache; this action triggers a syntax check of the configuration. The results will show, among other things, whether the module you wanted to disable is still referenced.

The use of a firewall – either centrally or directly in the operating system – is also useful. In this way you can limit, for example, the number of incoming connections per IP (connlimit with iptables; meters or dynamic sets with nftables). Also, it is often not necessary to allow all outgoing connections on a web server. For example, the iptables rule

# iptables -A OUTPUT -m owner --uid-owner www-data -m state --state new -j DROP

disallows outgoing traffic from the www-data system user that does not belong to any existing connection.

You can achieve additional security at the operating system level with SELinux (Red Hat, SUSE) or AppArmor (Debian, Ubuntu). Both extensions implement mandatory access control (MAC) and are extremely powerful, but they also require some training.

To be in a position to respond promptly to failures, you will want to monitor the web server with a monitoring solution like Icinga or Zabbix. Other useful extensions include intrusion prevention and detection systems (IPS/IDS) from the open source sector such as Suricata or OSSEC. Comprehensive security management also includes checking logfiles for potential attacks and, ideally, integrating them into a security information and event management (SIEM) system.

Configuration

Currently Apache 2.4 Core has 90 configuration directives [2]. In any case, you should be familiar with the Directory, Files, Limit, Location, and VirtualHost configuration groups. For those who have been using Apache for many years, it is best to take a look at the changes in version 2.4 [3], which include the new mod_authz_host access module with the Require directive, which replaces the previous Order/Allow/Deny syntax.

Without explicit configuration, the Directory directive allows access to the entire root filesystem, which can be prevented by the configuration shown in Listing 1. Depending on requirements, you can then proceed to assign more privileges to the subdirectories you actually use.

<Directory "/">
  Options None
  AllowOverride None
  Require all denied
</Directory>
<Directory "/var/www/html/">
  Require all granted
</Directory>

Enabling FollowSymLinks allows symbolic links to be followed on the filesystem. Remember that a symbolic link could be used to access the /etc/passwd file. A more secure variant here is SymLinksIfOwnerMatch, which only follows the link if the symlink and target owners are identical.

Options controls the functions available in a directory. Indexes affects the directory listing; Include and ExecCGI allow or disallow server-side includes with mod_include and CGI scripts with mod_cgi, respectively. A very restrictive configuration of the Options directive might be:

<Directory "/var/www/html/">
  AllowOverride None
  Options -Indexes -Includes -ExecCGI -FollowSymLinks
</Directory>

Options also support inheritance through the use of + and -. In the following configuration, FollowSymLinks and Includes are in effect for the /var/www/html/help directory:

<Directory "/var/www/html/">
  Options Indexes FollowSymLinks
</Directory>
<Directory "/var/www/html/help">
  Options +Includes -Indexes
</Directory>

The AllowOverride directive deserves special attention because it determines the handling of the very powerful .htaccess files, which None disables. You can use AuthConfig, FileInfo, Indexes, and Limit to allow certain settings. If direct access to the Apache configuration is possible, you generally want to avoid using .htaccess. It makes sense to separate the configuration from the content.

Like Directory, the Files directive can be used to impose restrictions for specific file names, as in the example in Listing 2, which prohibits downloading .htaccess and .htpasswd files. Limit lets you restrict access to certain HTTP methods. In this case, only a successfully authenticated user has access to the specified methods. The Location directive refers to the URL and should only be used when dealing with content outside the filesystem.

<Files ".ht*">
   Require all denied
</Files>
<Limit POST PUT DELETE>
   Require valid-user
</Limit>
<Location /status>
   SetHandler server-status
   Require ip 192.0.2.0/24
</Location>

Directory and Files are always processed first. DirectoryMatch, FilesMatch, and LocationMatch support the use of regular expressions, as in:

<FilesMatch "\.(gif|jpe?g|png)$">

Always pay attention to the context in which individual directives apply, as illustrated in the official documentation (Figure 1).

Security Issues

The ServerTokens and ServerSignature directives are often referred to in security discussions. In general, obfuscating the web server software or version number does not genuinely improve security, but you will definitely want to update. After all, you don't want to paint a target on your forehead because the web server reports an outdated version.

ServerSignature is set to Off by default, which means that Apache normally does not add a footer to the pages with Apache Server at www.example.com Port 443. In contrast, ServerTokens defaults to Full, and the resulting output is a header. You can query this header with wget -s (--server-response).

The usual recommendation is to use a different setting to avoid revealing too much information immediately about the web server. The minimal variant Prod means that the server only outputs the Apache name:

ServerSignature Off
ServerTokens Prod

The FileETag and TraceEnable directives also need to be disabled by setting them to None and off, respectively, for security reasons.

Fending Off DoS Attacks

The following directives are useful for hardening Apache against denial-of-service (DoS) attacks:

• RequestReadTimeout
• TimeOut
• KeepAliveTimeout
• LimitRequestBody
• MaxRequestWorkers (was MaxClients)
• MaxConnectionsPerChild (was MaxRequestsPerChild)

The timeout options affect how long Apache keeps connections open. Current Apache 2.4 versions usually use sensible defaults (e.g., in version 2.2, RequestReadTimeout was still set to 0).

LimitRequestBody also has a value of zero in the current Apache version, which means that a client is always allowed to transmit unlimited volumes of data. MaxRequestWorkers can be used to control how many simultaneous HTTP connections are allowed and should always be set to reflect the available RAM. MaxConnectionsPerChild does not have a limit by default. A limit can be useful to tell processes to release RAM in case of memory leaks.

HTTP Headers

By defining HTTP headers, you can enhance website security in several ways. On the one hand, you can do this in the HTML code and with server-side scripting languages. On the other hand, you could opt for a centralized approach through the Apache configuration. The examples in Listing 3 show possible application scenarios.

Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
Header always set Strict-Transport-Security "max-age=63072000"
Header always append X-Frame-Options SAMEORIGIN
Header always set X-XSS-Protection "1; mode=block"
Header always set X-Content-Type-Options nosniff
Header always set Content-Security-Policy "default-src 'mailto' 'self'"

The HttpOnly flag prevents cookies being read with scripting languages such as Java or VBScript. The Secure flag transmits the cookie only if an encrypted HTTPS connection exists. The Strict-Transport-Security header enables HTTP Strict Transport Security (HSTS) for a domain and ensures that, on future visits, the browser will always automatically call the specific domain by HTTPS and deny HTTP access. The remaining options are aimed at providing protection against cross-site scripting (XSS) and similar attacks on the browser.

Mozilla HTTP Observatory [4] provides a good overview of security measures already implemented on a website and suggests further recommendations (Figure 2).

TLS/SSL Configuration

With most browsers now warning users against unencrypted HTTP connections, HTTPS connections are already part of a web server's standard configuration.

Mozilla SSL Configuration Generator [5] is a good starting point for a sensible configuration: You specify the server software, including the exact Apache and OpenSSL versions, and then select the desired security level (Figure 3). The default is Intermediate, which is fine in most cases. If you want particularly high security, select Modern, which means that only TLS 1.3 is supported and it will surely lock out some visitors on busy sites. Optionally, you can enable HSTS and Online Certificate Status Protocol (OCSP) stacking; both are recommended.

The result is a prebuilt configuration (Figure 4) you can adopt into your own Apache settings. Before restarting to enable the configuration, you again need to check the syntax by typing apache2ctl -t.

The TLS Checklist Inspector [6] and SSL Labs [7] can help you check the TLS configuration. The German Checklist Inspector follows Technical Guidelines 03116, Part 4, of the German Federal Office for Information Security [8]. These guidelines are also a good source when it comes to serious recommendations on the subject of TLS/SSL.

Let's Encrypt with mod_md

The managed domain module, mod_md, entered Apache in version 2.4.30 and is available as of Ubuntu 20.04 and Debian 10. Additionally, the mod_watchdog module handles periodic tasks. You can use mod_md to integrate the Automatic Certificate Management Environment (ACME), which is responsible for issuing certificates directly in the Apache configuration. The simplest case will have syntax as in Listing 4. This configuration tells the server to obtain a new certificate automatically from Let's Encrypt, which the virtual host then uses. The renewal process is also automated. Which Let's Encrypt challenge takes effect depends on the port on which Apache is listening. For port 80, you can use the HTTP-01 challenge; otherwise, use TLS-ALPN-01 on port 443.

MDomain example.org
MDCertificateAgreement accepted
<VirtualHost *:443>
 ServerName www.example.com
 ServerAdmin hostmaster@example.com
 SSLEngine on
</VirtualHost>

A wildcard certificate requires a DNS-01 challenge, for which you need to specify a script (i.e., in /usr/bin/acme-setup-dns) that matches the DNS records:

MDChallengeDns01 /usr/bin/acme-setup-dns

This script has to support two calls:

acme-setup-dns setup example.com <challenge-data>
acme-setup-dns teardown example.com

You also need to provide a valid email address in the ServerAdmin directive, which is stipulated as a mandatory requirement by Let's Encrypt and which the certification authority uses for important notifications.

The Apache log reveals that a new certificate has been obtained:

[Fri Jan 01 13:41:06.880247 2020] [md:notice] [pid 776:tid 139942708430592] AH10059: The Managed Domainwww.example.com has been setup and changes will be activated on next (graceful) server restart.

Afterward, at least a graceful restart is required. When a certificate is renewed, an automatic restart with cronjob makes sense.

The mod_md module extends mod_status to include the md-status section, which means you get JSON output for a specific domain. To query this, use https://www.example.com/md-status/example.com. The special status page can be enabled with the configuration:

<Location "/md-status">
 SetHandler md-status
 Require ip 192.0.2.0/24
</Location>

In the default setting, a certificate status can also be retrieved with https://www.example.com/.httpd/certificate-status. You can disable access to it with MDCertificateStatus off.

If you don't want to use Let's Encrypt certificates, you can use the mod_md module for commercial certificate authorities. Some providers now support the standardized ACME protocol.

Web Application Firewall

The Apache mod_security2 module implements a full-fledged web application firewall (WAF). Because it is integrated directly into Apache, it also works independently regardless of whether a connection is encrypted with HTTPS. The WAF can manipulate the request or the web server's response directly. The module can therefore intervene and provide protection against an attack before the request reaches the web application.

On Ubuntu and Debian, the installation is very simple, thanks to a pre-built module in the repository. To install, run:

apt install libapache2-mod-security2

You then need a configuration file named /etc/modsecurity/modsecurity.conf. However, that's it for the easier part of the setup. The mod_security2 module is extremely powerful, so I can only provide a brief overview at this point.

By default, the module starts in detection mode (SecRuleEngine DetectionOnly), which allows the web application to continue working as usual so you can check ruleset matches. On Debian and Ubuntu, you can find the logfile in /var/log/apache2/modsec_audit.log.

You can create all the rules and allow and block lists. However, you also can find free and well-maintained default rules, such as the OWASP ModSecurity Core Rule Set. According to its own specifications, it offers protection against many different types of attacks [9].

Chroot Versus Containers

The mod_unixd module and the ChrootDir directive let you run Apache in a chroot environment. For a basic static web server, this is quite simple to achieve.

Things gets more complex if you want to use scripting languages in the chroot, too, and will work with mod_php, even in the chroot environment. However, you will face restrictions with certain functions (e.g., the mail command), for which you then need additional libraries and configuration files in the chroot directory, quickly making the whole project more error-prone.

As an alternative to the PHP module, the PHP-FPM FastCGI variant comes with its own chroot option. The Apache mod_security2 module offers a SecChrootDir directive for the same purpose, but the overhead remains similar.

Independent of the chroot option, isolation in containers is also a good idea. Technology-wise, you can use either Docker containers or Linux containers (LXC/LXD), which allow separate Apache instances for individual applications. If you have to save IP addresses with virtual hosts, it can make sense to connect an upstream proxy that can handle Server Name Indication (SNI) for HTTPS connections in such a scenario.

Conclusions

Sophisticated configuration, timely updates, and well-thought-out security concepts cannot be implemented in the blink of an eye; instead, they require the admin's constant attention. If you follow all the tips in this article on securing your web server, you will not have to worry too much – the floodgates will be leak-tight.