Key Cabinet

At OpenVPN seminars, participants arrive with their own wishes and ideas. Although they are satisfied with the OpenVPN support for bandwidth limitation, simple high availability, and flexible traffic management, for example, one topic remains unclear and concerns all VPNs: How does the admin create and manage a simple secure sockets layer (SSL) public key infrastructure (PKI) for many users without spending a large amount of cash on service providers or proprietary software? The ideal solution would be open source – free of licensing costs and similar complications and definitely not cloud- or web-service-based – in which the use of self-signed certificates is not a problem.

In-House Label

OpenVPN [1] traditionally relies on Easy-RSA [2], of which version 2 is widely used. Many, especially the Debian-based, distributions install it along with openvpn – one exception being Ubuntu, which only offers easy-rsa starting with Cosmic Cuttlefish (Ubuntu version 18.10) [3].

The successor, Easy-RSA 3.0 [4], has been available for years and simplifies a few things but is not that different from version 2, on which most solutions and setups are still based. Both versions have one thing in common: They come without a fancy GUI, but as plain vanilla command-line tools, which is a bit unusual for point-and-click GUI users and many admins.

Although Open CA, XCA, and TinyCA also are open source tools for making PKI administration easier, none of them has had any genuine success. The topic itself seems to be too complex and too prone to error, with a workflow that users and even admins just cannot comprehend.

OpenVPN founder James Yonan of OpenVPN Technologies obviously also recognized this problem. Among other things, the software is supported by a web front end that eases the generation and distribution of certificates. However, this is only true for the company's own OpenVPN product, which the company will be releasing as OpenVPN 3.0 at some point.

Armed with the knowledge from my OpenVPN seminars, I can safely say that creating the first set of server, client, and CA certificates and keys are the most difficult tasks, regardless of the tool used, because in the end, OpenSSL always works in the background with command lines like that shown in Listing 1.

openssl x509
  -req
  -CA ca.crt
  -CAkey ca.key
  -set_serial 0x$(openssl rand -hex 16)
  -days 3650
  -extfile <(echo -e "keyUsage=digitalSignature,keyEncipherment\nextendedKeyUsage= serverAuth")
  -in server.csr
  -out server.crt

Because no admin wants to type this stuff on a regular basis, many instructions and scripts simplify the process without significantly increasing the error rate. Easy-RSA is also basically only an intelligent OpenSSL wrapper.

I asked friends and acquaintances in the Linux and security world whether this would work for thousands of users. It does, replied – among others – the experienced sys admins at Berlin's CharitÈ [5], Germany's largest hospital.

The CharitÈ admins have extended Easy-RSA by adding a few scripts and currently manage 17,000 users. As Ralf Hildebrandt, Senior Network Engineer at CharitÈ and often a helpful point of contact, explained: "We use Easy-RSA on the VPN server and automatically generate user certificates in the form <Username>.vpn.charite.de. If a user leaves, we revoke the certificate after a certain grace period. We get the login data from LDAP."

Easy-RSA 3

In openSUSE Tumbleweed, the command

zypper in easy-rsa

installs version 3.0 of the software. You can accomplish the same with Apt or Aptitude, Yum, or the package manager of your choice on other distributions. In most cases, however, this command will only install an earlier version.

After the install, you find the configuration files in the /etc/easy-rsa/ directory; working as root in this directory, call all the following commands.

easyrsa init-pki
easyrsa gen-dh
easyrsa build-client-full <Unique_DN_of_Client_Certificate> nopass

Just these few steps take you to your own PKI. The first command initializes the PKI, deletes all previously entered data, if necessary, and creates a Certificate Authority (CA). Watch out, this completely resets the PKI. If you want to make your work easier, edit the vars file and add your data to save some typing later. The second command creates a Diffie-Hellman key for the initial key exchange between unknown, untrusted partners, and the third creates a client certificate that provides access to the VPN without a password. For example, if the last command reads

easyrsa build-client-full Client234 nopass

then Client234 appears as a distinguished name (DN) entry in the client certificate; OpenVPN uses this name for numerous practical functions. Similarly,

easyrsa build-server-full <DN> nopass

generates a server certificate without password protection. Now you can find certificates and private and public Diffie-Hellman keys in the directories under /etc/easy-rsa/pki (Figures 1 and 2).

You will want to isolate this PKI structure from the OpenVPN server – not least because these folders contain all the private keys for all the certificates. Ideally, Easy-RSA should run on an isolated system without a connection to the Internet. Although this process might seem archaic and cannot always be put into practice, losing the private key of the PKI (ca.key) opens up the whole infrastructure to attackers.

By the way,

easyrsa export-p12 <DN>

exports a key pair in PKCS12 format. The

easyrsa gen-req <DN>
easyrsa sign-req <DN>

commands give Easy-RSA full-fledged PKI management, and the

build-client-full
build-server-full

commands handle the request and sign procedure automatically, without requiring any intervention on the part of the admin.

The scripts CharitÈ uses are based on Easy-RSA 2.2.2-2, which comes with the Debian version they use in Berlin. However, they can be put together quite quickly for version 3. Further details (see Listing 2 with English translations) were provided by Hildebrandt: "Our script is named /opt/openvpn/scripts/generate_certs_for_active_users.py; it generates certificates for the active users on the VPN system starting in lines 47 and 60 if no certificate is available yet. A search against /etc/easy-rsa/ with ll*.crt | wc -l currently finds 16,849 user certificates."

01 #!/usr/bin/python
02 import urllib, json, os, subprocess, sys
03
04 url = "http://vpnapi.charite.de/vpn/user/status/activevpn"
05 response = urllib.urlopen(url);
06 data = json.loads(response.read())
07
08 for userentry in data:
09     if userentry.get('active'):
10         username = userentry.get('username')
11
12         if os.path.isfile( "/opt/openvpn/ca/keys/" + username + ".crt"):
13             # print "Cert for " + username + " exists"
14             a=1
15         else:
16
17             # get user details
18             detailsurl = "http://vpnapi.charite.de/vpn/user/details/" + username
19             response = urllib.urlopen(detailsurl);
20             detaildata = json.loads(response.read())
21             useremail = detaildata.get('mail')
22             if not useremail:
23                 print "User " + username + " has no Email address in LDAP"
24                 # Import smtplib for the actual sending function
25                 import smtplib
26
27                 # Import the email modules we'll need
28                 from email.mime.text import MIMEText
29
30                 # Create a text/plain message
31                 msg = MIMEText("User " + username + " has no Email address in LDAP")
32
33                 [...]
34                 msg['Subject'] = 'User %s has no Email address in LDAP' % username
35                 msg['From'] = 'vpn@charite.de'
36                 msg['To'] = 'vpn@charite.de'
37
38                 [...]
39                 s = smtplib.SMTP('localhost')
40                 s.sendmail('vpn@charite.de', 'vpn@charite.de', msg.as_string())
41                 s.quit()
42                 break
43
44             # end get user details
45
46             print "Need to generate cert for " + username
47             p = subprocess.Popen("/opt/openvpn/ca/build-key-auto " + username, bufsize=1, cwd="/opt/openvpn/ca", stdout=subprocess.PIPE, close_fds=False, universal_newlines=True, shell=True)
48             if p.wait:
49                 ret = p.wait()
50             else:
51                 ret = p.returncode
52             if (ret != 0):
53                 # if not OK
54                 if p.stdout:
55                     print "Output was:"
56                     print p.stdout.readlines()
57             else:
58                 # if OK
59                 print "Send Certificate!"
60                 p = subprocess.Popen("/opt/openvpn/scripts/send_certs_and_config.py " + username, bufsize=1, cwd="/opt/openvpn/ca", stdout=subprocess.PIPE, close_fds=False, universal_newlines=True, shell=True)
61                 if p.wait:
62                     ret = p.wait()
63                 else:
64                     ret = p.returncode
65                 if (ret != 0):
66                     if p.stdout:
67                         print "Output was:"
68                         print p.stdout.readlines()
69                         # if not OK

The small collection contains two longer and three short scripts (available online [6]):

• generate_certs_for_active_users.py creates a certificate for all existing users without a certificate (Listing 2).
• send_certs_and_config.py (Listing 3) signs them with S/MIME and mails to the VPN users.
• revoke_remove_cert_without_user (Listing 4) is applied when a user leaves.
• checkCertWithoutUser.pl and revoke_and_delete (called in Listing 4, line 2) [6] get rid of certificates after some time has elapsed (Figure 3).

The next step is to take a detailed look at the listings.

Certs for Active Users

To create a certificate for the active users of the VPN system, generate_certs_for_active_users.py (Listing 2) uses the JSON API to query which users are active. It iterates over the users and checks whether /opt/openvpn/ca/keys contains a .crt file with the name of the respective user. If this is not the case, it generates a certificate with the /opt/openvpn/ca/build-key-auto username script provided with Easy-RSA. Then, send_certs_and_config.py (Listing 3) mails the signed certificates to the users. The VPN server only needs to recognize the CA; the PKI work can and should be done elsewhere.

001 #!/usr/bin/python
002 # -*- coding: utf-8 -*-
003 import urllib, json, subprocess, argparse, re, sys
004 import [...]
005
006 from M2Crypto import BIO, Rand, SMIME
007 from [...]
008
009 [...]
010 ssl_key  = '/etc/ssl/private/mailcert-vpn.charite.de.key'
011 ssl_cert = '/etc/ssl/certs/mailcert-vpn.charite.de.crt'
012
013
014 def send_mail_ssl(sender, to, subject, text, files=[], attachments={}, bcc=[]):
015     """
016     Sends SSL signed mail
017     [...]
018     """
019
020     if isinstance(to, str):
021         to = [to]
022
023     # create multipart message
024     msg = MIMEMultipart()
025
026     # attach message text as first attachment
027     msg.attach( MIMEText(text, "plain", "utf-8") )
028
029     # attach files to be read from file system
030     for file in files:
031         part = MIMEBase('application', "octet-stream")
032         part.set_payload( open(file,"rb").read() )
033         Encoders.encode_base64(part)
034         part.add_header('Content-Disposition', 'attachment; filename="%s"'
035              % os.path.basename(file))
036         msg.attach(part)
037
038     # attach files read from dictionary
039     for name in attachments:
040         part = MIMEBase('application', "octet-stream")
041         part.set_payload(attachments[name])
042         Encoders.encode_base64(part)
043         part.add_header('Content-Disposition', 'attachment; filename="%s"' % name)
044         msg.attach(part)
045
046     # put message with attachments into SSL' I/O buffer
047     msg_str = msg.as_string()
048     buf = BIO.MemoryBuffer(msg_str)
049
050     # load seed file for PRNG
051     Rand.load_file('/tmp/randpool.dat', -1)
052
053     smime = SMIME.SMIME()
054
055     # load certificate
056     smime.load_key(ssl_key, ssl_cert)
057
058     # sign whole message
059     p7 = smime.sign(buf, SMIME.PKCS7_DETACHED)
060
061     # create buffer for final mail and write header
062     out = BIO.MemoryBuffer()
063     out.write('From: %s\n' % sender)
064     out.write('To: %s\n' % COMMASPACE.join(to))
065     out.write('Date: %s\n' % formatdate(localtime=True))
066     out.write('Subject: %s\n' % subject)
067     out.write('Auto-Submitted: %s\n' % 'auto-generated')
068     out.write('X-Auto-Response-Suppress: %s\n' % 'OOF')
069
070     # convert message back into string
071     buf = BIO.MemoryBuffer(msg_str)
072
073     # append signed message and original message to mail header
074     smime.write(out, p7, buf)
075
076     # load save seed file for PRNG
077     Rand.save_file('/tmp/randpool.dat')
078
079     # extend list of recipients with bcc addresses
080     to.extend(bcc)
081
082     # finally send mail
083     p = Popen(["/usr/sbin/sendmail", "-fvpn@charite.de", "-oi", COMMASPACE.join(to)], stdin=PIPE, universal_newlines=True)
084     p.communicate(out.read())
085
086 # BEGINNING OF SCRIPT
087 if not os.geteuid() == 0:
088     sys.exit('Script must be run as root')
089
090 # Parse command-line arguments
091 parser = argparse.ArgumentParser(description='Sends VPN configuration file to users')
092
093 parser.add_argument('userinput', help='<username|mailaddress>')
094 parser.add_argument('mailaddress', nargs='?', help='<Destination mail address - default is the user's request email address>')
095 args = parser.parse_args()
096
097 [...]
098
099 is_mail = re.compile('^.+?\@.+')
100 if is_mail.match(args.userinput):
101     #print args.userinput + " is an email address"
102     url = "http://vpnapi.charite.de/vpn/user/detailsbyemail/" + args.userinput
103 else:
104     #print args.userinput + " is a username"
105     url = "http://vpnapi.charite.de/vpn/user/details/" + args.userinput
106
107 #response = urllib.urlopen(url);
108 response = urllib.urlopen(url, proxies={});
109 data = json.loads(response.read())
110
111 username = data.get('username')
112 active = data.get('vpnActive')
113
114 if args.mailaddress:
115     if is_mail.match(args.mailaddress):
116         #print args.mailaddress + " is an email address"
117         email = args.mailaddress
118     else:
119         print args.mailaddress + " is an email address!"
120         sys.exit(1)
121 else:
122     email = data.get('mail')
123
124 if not username:
125     print "Username does not exist"
126     sys.exit(1)
127
128 if not active:
129     print "User is not active"
130     sys.exit(1)
131
132 if not email:
133     print "Email is blank"
134     sys.exit(1)
135
136 # Encode email address if not blank!
137 email = email.encode('ascii','ignore')
138
139 #print username, email
140
141 if os.path.isfile( "/opt/openvpn/ca/keys/" + username + ".crt"):
142     #print "Cert for " + username + " exists"
143
144     # Read in the text portion of the mail
145     with open('/opt/openvpn/etc/sendcertagain-mailbody.txt', 'r') as myfile:
146         text=myfile.read()
147
148     # Set filenames for the attachment
149     configfilename="charite-" + username + ".ovpn"
150
151     # Define key material paths
152     keyfilename="/opt/openvpn/ca/keys/" + username + ".key";
153     crtfilename="/opt/openvpn/ca/keys/" + username + ".crt";
154     cafilename="/opt/openvpn/ca/keys/ca.crt";
155
156     # Open key material
157     with open('/opt/openvpn/etc/charite.ovpn.template', 'r') as myfile:
158         configtemplatefile = myfile.read()
159     with open(cafilename, 'r') as myfile:
160         cafile = myfile.read()
161     with open(keyfilename, 'r') as myfile:
162         keyfile = myfile.read()
163     with open(crtfilename, 'r') as myfile:
164         crtfile = myfile.read()
165
166     sender = str(Header("CharitÈ VPN administrator", "utf8")) + " <vpn@charite.de>"
167     subject = Header("Configuration file to your CharitÈ VPN access","utf8")
168
169     configfile = configtemplatefile + "<ca>\n" + cafile + "</ca>\n<key>\n" + keyfile + "</key>\n<cert>\n" + crtfile + "</cert>\n"
170     send_mail_ssl(sender, email, subject, text, files=[], attachments={configfilename: configfile} )
171     print "Configuration file was sent to " + email + "."
172
173 else:
174     print "User " + username + " has no certificate"
175     sys.exit(1)
176 if os.path.isfile( "/opt/openvpn/ca/keys/" + username + ".crt"):
177     #print "Cert for " + username + " exists"
178
179     # Read in the text part of the mail
180     with open('/opt/openvpn/etc/sendcertagain-mailbody.txt', 'r') as myfile:
181         text=myfile.read()
182
183     # Set attachment filenames
184     configfilename="charite-" + username + ".ovpn"
185
186     # Define key material paths
187     keyfilename="/opt/openvpn/ca/keys/" + username + ".key";
188     crtfilename="/opt/openvpn/ca/keys/" + username + ".crt";
189     cafilename="/opt/openvpn/ca/keys/ca.crt";
190
191     # Open key material
192     with open('/opt/openvpn/etc/charite.ovpn.template', 'r') as myfile:
193         configtemplatefile = myfile.read()
194     with open(cafilename, 'r') as myfile:
195         cafile = myfile.read()
196     with open(keyfilename, 'r') as myfile:
197         keyfile = myfile.read()
198     with open(crtfilename, 'r') as myfile:
199         crtfile = myfile.read()
200
201     sender = str(Header("CharitÈ VPN Administrator", "utf8")) + " <vpn@charite.de>"
202     subject = Header("Configuration file to your CharitÈ VPN access","utf8")
203
204     configfile = configtemplatefile + "<ca>\n" + cafile + "</ca>\n<key>\n" + keyfile + "</key>\n<cert>\n" + crtfile + "</cert>\n"
205     send_mail_ssl(sender, email, subject, text, files=[], attachments={configfilename: configfile} )
206     print "Configuration file was sent to " + email + "."
207
208 else:
209     print "User " + username + " has no Certificate"
210     sys.exit(1)

On Revocation

When employees leave their employer, admins need to make sure they prevent further VPN access. At the CharitÈ, this is done with the revoke_remove_cert_without_user script (Listing 4), which uses checkCertWithoutUser.pl to generate a list of certificates for which active users are missing and pipes this list to revoke_and_delete, which Easy-RSA uses to revoke and delete the key material (Figure 3). The certificates are only irreversibly deleted after a transitional period of three months, because "often users come back within three months," explained Hildebrandt, "in which case, they don't want to impose the burden of having to install a new configuration or new certificates."

01 #!/bin/sh
02 /opt/openvpn/scripts/checkCertWithoutUser.pl | xargs --no-run-if-empty --replace /opt/openvpn/scripts/revoke_and_delete {}

There You Go!

According to Hildebrandt, the CharitÈ system, which now manages 17,000 users, surprised even the administrators: Working with Easy-RSA is smooth and stable in enterprise operation. "The advantage of Easy-RSA is clearly in its stability: the thing simply does exactly what you tell it to do – 100% and reliably," said Hildebrandt. "In more than 10 years of operation, it has never caused us trouble and always provided exactly the high-level commands we need to generate and withdraw certificates."

The configurations generated in this way also work with mobile devices and the practical OpenVPN format of the embedded keys. The configuration, certificates, and keys can be inserted directly into the configuration file without reference to other files, so users only have one configuration file for access, which significantly increases acceptance. This setup works fine with modern smartphones, as well.

Private keys that are not password protected are less critical: "Password protection during access is achieved via LDAP authentication, which is linked to Active Directory," explained Hildebrandt. "Every user has to enter their password anyway when they log in. Although this is the most frequently mentioned annoyance for users, it is necessary."

Additionally, neither Android nor iOS allow a web proxy via autoconfig. "Our users can use VPN, but the main purpose is to surf the web through our proxies, because they get full access to scientific journals and papers," said Hildebrandt. With Chrome OS, you can set exactly one proxy for a VPN connection.