Projects

An AWS VPN - Part 2

In Part 1 we got the EC2 instance up and working. Here we arduously walk through setting up OpenVPN. It's really not so bad...

AJ

Mar 23, 2019 • 15 min read

In Part 1 we got the EC2 instance up and working. Here we arduously walk through setting up OpenVPN. It's really not so bad, just long to type up...

OpenVPN

Finally we're on our EC2 instance. Let's get to configuring. I'm persnickety so I do updates first. Do a sudo apt update && sudo apt dist-upgrade and do the updates. If I get any linux headers, I sudo reboot next. Probably gonna get yelled at by some folks because it's probably not needed. After the reboot I do a cleanup: sudo apt autoclean && sudo apt autoremove and then if I clean up linux headers, I reboot again -- again, I'm probably gonna get yelled at for doing unnecessary things.

For the OpenVPN configuration I am mashing up a couple guides. If you're tired of my writing style, Google around for a guide or check out Digital Ocean's tutorial, which I follow pretty closeley here: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-openvpn-server-on-ubuntu-18-04

First up, get the package we'll need:

sudo apt install openvpn

PKI Setup

OpenVPN uses public key infrastructure (PKI) by default for the credentials to access the VPN. In order to leverage that we need to set up a PKI.

First let's get the right EasyRSA version. EasyRSA makes this PKI setup easier. I'd normally tell you to install the package via apt, but it's not working as I'm writing this due to it being out of date and and in the interest of being up to date... Download from GitHub:

wget -P ~/ https://github.com/OpenVPN/easy-rsa/releases/download/v3.0.6/EasyRSA-unix-v3.0.6.tgz
cd ~
tar xvf EasyRSA-unix-v3.0.6.tgz

You'll notice the version is hard-coded above. Please replace with the current version. You can see the current version by checking the EasyRSA GitHub release page: https://github.com/OpenVPN/easy-rsa/releases. Select the .tgz file for the most recent version and replace it both places in the above.

Next we want to go in and update the certificate information, first copy the example:

cd ~/EasyRSA-v3.0.6/
cp vars.example vars

Next we want to update the vars file (use your values, please). Uncomment and change the following lines:

set_var EASYRSA_REQ_COUNTRY     "US"
set_var EASYRSA_REQ_PROVINCE    "DC"
set_var EASYRSA_REQ_CITY        "Washington, DC"
set_var EASYRSA_REQ_ORG         "Org Name"
set_var EASYRSA_REQ_EMAIL       "you@example.com"
set_var EASYRSA_REQ_OU          "MyVPN"

Next we want to build the certificate authority (CA) for our PKI.

./easyrsa init-pki
./easyrsa build-ca nopass

If you want to build the CA to be NOT password protected, add nopass after the build-ca line, as above. Otherwise omit it.

Hit ENTER to accept the value for common name (CN), or update it. Then copy the CA certificate to the OpenVPN directory.

sudo cp pki/ca.crt /etc/openvpn/

Server certificate stuff

That built us the CA. Next we build the server certificate:

./easyrsa gen-req server nopass
sudo cp ~/EasyRSA-v3.0.6/pki/private/server.key /etc/openvpn/

This will create the CA signing request and copy the private key to the openvpn directory. Next we have to approve the request (type "yes" when prompted) and move the newly-issued server certificate:

./easyrsa import-req /tmp/server.req server
./easyrsa sign-req server server
sudo cp pki/issued/server.crt /etc/openvpn/

Next we need to generate a Diffie-Hellman key which is used during initial key exchange. Give this some time to run. Then generate a TLS authorization key. Then copy those keys into the OpenVPN directory:

./easyrsa gen-dh
openvpn --genkey --secret ta.key
sudo cp ta.key /etc/openvpn/
sudo cp pki/dh.pem /etc/openvpn/

Client certificate stuff

Ok now we need to setup client certificates. Assuming you're going to have multiple clients, like 1 for your phone and 1 for your laptop, it'll be easier if they have their own directory. Below we make that, change permissions on the folder, and generate a client request, copy the key to the keys directory, and sign the request, and copy the certs and keys clients need to the client config directory:

mkdir -p ~/client-configs/keys
chmod -R 700 ~/client-configs

cd ~/EasyRSA-v3.0.6/
./easyrsa gen-req client1 nopass

cp pki/private/client1.key ~/client-configs/keys/

./easyrsa sign-req client client1

cp pki/issued/client1.crt ~/client-configs/keys/
cp ta.key ~/client-configs/keys/
sudo cp /etc/openvpn/ca.crt ~/client-configs/keys/

You can perform the client certificate generation process any number of times. It's best practice to not reuse these client certificates across devices.

Configure the OpenVPN service

The first thing we'll do is copy a sample configuration for OpenVPN. It just seems easier...

sudo cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/
sudo gzip -d /etc/openvpn/server.conf.gz

Next up we'll edit the /etc/openvpn/server.conf file. I will try to keep the edits in order here:

Find the section that gives the port number. It will be defaulted to port 1194, but like I mentioned in Part 1 change it to port 443 because some free wifi places aren't default-port friendly. I've never had a problem with 443 since that's shared with HTTPS.
In the next section down, comment out the proto udp line by adding a semicolon, and remove the semicolon from proto tcp. This should make it more compatible with places that block UDP for some reason (I experienced this once).
Find the section for Diffie-Hellman, which starts with dh. Next change the filename in that directive from whatever is there (usually dh2048.pem) to the file we created earlier in the tutorial. In our case just rename it to dh.pem and you're good.
Find the section that says ;push "redirect-gateway def1 bypass-dhcp" and uncomment it by removing the semicolon. This will direct all traffic at the client to use the VPN which is our desired behavior.
Right below that, find the lines ;push "dhcp-option DNS 208.67.222.222" and ;push "dhcp-option DNS 208.67.220.220". Uncomment those as well by removing the semicolon. This will tell the client to use OpenDNS for DNS queries. You can change to your preferred DNS provider. Google Public DNS (8.8.8.8 and 8.8.4.4) and Cloudflare's DNS service (1.1.1.1 and 1.0.0.1) are both good options to put here if you don't want to use OpenDNS.
Find the section for tls-auth and underneath the line that says tls-auth ta.key 0, add a line that says: key-direction 0. The comments above explain that "0" is used on the server and "1" on the clients.
A few lines below that, find the line that says cipher AES-256-CBC and again below that, add a line that says: auth SHA256. This line specifies how to authenticate using that ta.key (I'm pretty sure)
Find the lines that say ;user nouser and ;group nogroup. Uncomment those by removing the semicolons.
On the last line of the config file, find the line explicit-exit-notify and change it to read explicit-exit-notify 0.
Save the file and close it.

Alright. That's all the changes for the OpenVPN configuration. Next up...

Network Configurations

We need to configure the server to forward IP packets from our clients. To do that, sudo edit the /etc/sysctl.conf. Find the line net.ipv4.ip_forward=1 and uncomment it by deleting the #. Save and close the file. That will change the behavior in the future, but we also need to change it for our running config:

sudo sysctl -p

We want to make sure we have a firewall active, so let's activate UFW (uncomplicated firewall). This should have been installed already with the EC2 AMI we chose. If not, install it.

sudo ufw enable

sudo ufw allow OpenSSH

sudo ufw status

Now we can make changes to let the VPN work. First get the interface name by running:

ip route | grep default

The output will show a bunch of info, but look between "dev" and "proto" to get your interface name. Make note of this (mine is named ens5)

Next up open the /etc/ufw/before.rules file and edit the beginning of the file to look like below. DO NOT change the rest of the file. Only add the chunk between # START OPENVPN RULES and # END OPENVPN RULES.

#
# Rules that should be run before the ufw command line added rules. Custom
# rules should be added to one of these chains:
#   ufw-before-input
#   ufw-before-output
#   ufw-before-forward
#

# START OPENVPN RULES
# NAT table rules
*nat
:POSTROUTING ACCEPT [0:0]
# Allow traffic from OpenVPN client to wlp11s0 (change to the interface you discovered!)
-A POSTROUTING -s 10.8.0.0/8 -o ens5 -j MASQUERADE
COMMIT
# END OPENVPN RULES


# Don't delete these required lines, otherwise there will be errors

Next up edit the UFW configuration to allow packet forwarding by editing /etc/default/ufw. Change the DEFAULT_FORWARD_POLICY directive to DEFAULT_FORWARD_POLICY="ACCEPT".

Ok next we need to open up the firewall to allow the traffic for the VPN.

sudo ufw allow 443/tcp

If you didn't change port or protocol earlier in the OpenVPN configuration, then choose whatever you chose.

Finally done with network configuration. Next up is...

Starting OpenVPN

Finally we're ready to start up OpenVPN. We will use systemctl right afterwards to make sure the service started correctly

sudo systemctl start openvpn@server
sudo systemctl status openvpn@server

(hit q to escape)

If everything worked (fingers crossed), you should see that the service is active. If not, read the error message and try the command journalctl -xe to read what failed.

Next up we want to ensure it starts automatically at next startup.
sudo systemctl enable openvpn@server

(D)DNS?

Ok. So before we set up client certificates, we should ensure we can reach our EC2 instance VPN.

One of the reasons I like using EC2 is that I don't have to pay for my instance if I know I won't be using it. However, the way AWS works is if you reserve a public IP (called an Elastic IP) it's only free as long as you're using it. Once you turn off the instance it's attached to, you start getting charged for it. This is not ideal for me, so I want to leverage Dynamic DNS (DDNS) in order to be able to reach my VPN.

Enter Duck DNS. The good folks over at Duck DNS have made it super easy to get DDNS up and running for free. (Support them via donation at their website). Other providers exist, but I'm going to use Duck DNS here. Set up an account with them by following the directions at their site. And once you're ready and have your token, come back here. I'll wait...

I want to automate the DDNS update process to run every 5 minutes. That way I only ever have to wait a maximum of 5 minutes for my VPN to become available for use after boot.

First make a directory for our Duck DNS stuff, add the script we'll make to perform this for us, and make it executable:

mkdir ~/duckdns
touch ~/duckdns/duck.sh
chmod +x ~/duckdns/duck.sh

Edit the duck.sh file to match the below, but put in your domain and your token from Duck DNS:

#!/bin/bash

curl -o /home/ubuntu/duckdns/duck.log "https://www.duckdns.org/update?domains={YOURVALUE}&token={YOURVALUE}"

echo "" >> /home/ubuntu/duckdns/duck.log

Save and close the file. Then add an entry to your /etc/crontab file. Add the following line to the file:

*/5 * * * * ubuntu /home/ubuntu/duckdns/duck.sh >/dev/null 2>&1

This will run our script every 5 minutes and throw away the output. Remember we made our own duck.log file in the script so we don't need the curl output filling our logs every 5 minutes.

In the Duck DNS webpage, hopefully we should see our new domain has an IP (if it's been 5 minutes). Additionally we should see the duck.log file has an updated time every 5 minutes.

Now that we have DDNS set up. Let's move on to...

Client Configurations

I'm going to totally take this from the Digital Ocean tutorial because it should make the write-up so much easier. This section we're going to make a script which will create the client configurations as needed. This should make it easier to create configurations later and also reduces the number of edits we need to make in a single config file each time we send it to a client. Let's jump in.

NOTE: if you don't want to make a script, just continue following directions below until you get to step #4 below. Configure the client.crt and client.key names to their corresponding names we created in the PKI section above.

We start by making a directory for the client configuration files. Then we will copy the OpenVPN sample client configuration there:

mkdir -p ~/client-configs/files
cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf ~/client-configs/base.conf

Open up that file and edit it as described below:

In the proto directive area, uncomment tcp and comment out the udp line to match the server config. It should now read proto tcp and ;proto udp.
In the remote section a few lines down, edit it so it reads your DDNS name you just set up, so the client can always find the server. Also edit the port number here to match what we configured on the server, port 443. It should read something like remote {your-servername}.duckdns.org 443.
A few lines further down, uncomment (remove the semicolon) the user nouser and group nogroup directives.
Find the section with the ca, cert, and key lines. Comment those out by adding a # in front. So that it reads: #ca ca.crt, #cert client.crt, and #key client.key respectively. These lines we'd have to change for each client, and we will append these to our script-created file. NOTE: if you're not scripting, change them to the appropriate values and move on.
Scroll down a little further and comment out the tls-auth line by adding a # in front. We are going to add it directly into the config later. It should read #tls-auth ta.key now.
Underneath the cipher AES-256-CBC line add the following, to match the server config: `auth SHA256'.
Add to the end of the file a buffer line or two.
Add to the end of the file this line (or anywhere, but why not, since we're here): key-direction 1.

If you may have a linux client in the future, add these lines to the end, as shown:

#######################
# If the client is linux,uncomment these lines after the configuration is built
#######################
# script-security 2
# up /etc/openvpn/update-resolv-conf
# down /etc/openvpn/update-resolv-conf

Save and close the file

Now we'll make the script file. Create a file: touch ~/client-configs/make_config.sh. And edit it to contain the following:

#!/bin/bash

# First argument: Client identifier

KEY_DIR=~/client-configs/keys
OUTPUT_DIR=~/client-configs/files
BASE_CONFIG=~/client-configs/base.conf

cat ${BASE_CONFIG} \
    <(echo -e '<ca>') \
    ${KEY_DIR}/ca.crt \
    <(echo -e '</ca>\n<cert>') \
    ${KEY_DIR}/${1}.crt \
    <(echo -e '</cert>\n<key>') \
    ${KEY_DIR}/${1}.key \
    <(echo -e '</key>\n<tls-auth>') \
    ${KEY_DIR}/ta.key \
    <(echo -e '</tls-auth>') \
    > ${OUTPUT_DIR}/${1}.ovpn

I'm just going to straight up take the description of the above from Digital Ocean... I can't think of a better, more succinct way to describe it:

This script will make a copy of the base.conf file you made, collect all the certificate and key files you’ve created for your client, extract their contents, append them to the copy of the base configuration file, and export all of this content into a new client configuration file. This means that, rather than having to manage the client’s configuration, certificate, and key files separately, all the required information is stored in one place. The benefit of this is that if you ever need to add a client in the future, you can just run this script to quickly create the config file and ensure that all the important information is stored in a single, easy-to-access location.

Next make the file executable: chmod +x ~/client-configs/make_config.sh. Now we can use it.

cd ~/client-configs
sudo ./make_config.sh client1

If you made other clients, run the script again in the same way, but change client1 to the name of the client you configured in the PKI section.

NOTE: These .ovpn files the script creates contain private key information which need to be kept, well, private. Secret. Do not share these.

Because they have private key information we need a secure way to copy them to our clients. I like SCP, but SFTP works also. If we're using Windows we can SCP with pscp.exe, part of the PuTTY zip archive we could have downloaded in Part 1. If you didn't get pscp.exe then, go back to the PuTTY website and get the most up to date version.

On your local (Windows, in this example) machine, open up PowerShell and navigate to wherever you have your pscp.exe file located. Mine is in "Downloads" still, which is handy because that's where I want to download my .ovpn configurations also. Also, go find that .ppk file we made way back in Part 1. We will need it to authenticate.

Then run the following command: .\pscp.exe -i {your-ppk-file-} ubuntu@{your-DDNS-name}:~/client-configs/files/client1.ovpn .\

Run that again on your other clients to get their .ovpn configurations.

Connect to the VPN

Alright finally we're here. We should have all the configurations set up, the server is ready, and the client... oh.

Ok, go to the OpenVPN downloads page (https://openvpn.net/community-downloads/) and download the most recent OpenVPN client. I'm using Windows so I downloaded the Windows installer.

Install it. In the usual manner.

Start it. In the usual manner (it should drop a shortcut on the desktop, or we can get to it from the start menu).

Hopefully we should notice a new icon in the system tray:

Note the new OpenVPN icon

Right click that dude and select "import file." Navigate to where you've put your .ovpn client configuration (mine is still in "Downloads"). This import process copies the file to the default "Configuration Files" location in the OpenVPN settings: C:\Users\{your-username}\OpenVPN\config.

Note: there is a folder for .ovpn configurations in the C:\Program Files\OpenVPN\config\ directory also if you want to manually put it/them there.

Ok. Now we can test the VPN. Right-click on that icon again, and go to your newly-imported configuration and tell it to connect. A pop-up shows up with particulars during the setup phase. If all goes well, that window goes away and your icon turns green.

So green

We should be connected to the VPN. We can go to Google and search "my ip" and we should see the IP of our VPN server, not our home's IP. We can check also by trying IPLeak.net or DNSLeakTest. We should see your server's IP and the DNS servers should be the ones we configured. Remember we used OpenDNS, so we see a bunch of their DNS servers listed on those pages.

Disconnect the VPN

To disconnect, we right-click again and tell the OpenVPN GUI to disconnect from our profile.

Shut Down the Server (Optional)

We can shut down the server to save money while we're not using it. We still have to pay for storage each month (8GB), but won't have to pay for the EC2 instance itself while it's powered off. We can turn it off via the AWS console by selecting the instance and then selecting Actions > Instance State > Stop. Or by right clicking. We could also shut it down from the SSH session by issuing sudo poweroff.

Each time we know we'll want to use the VPN, like before traveling, we can just log back into the AWS console and start the instance. It should be easy to find because we tagged it. It may take up to 5 minutes to get the new DDNS address, don't forget, so be patient or risk poisoning your DNS cache with a bad IP and then we'll have to wait for the TTL.

After we start the instance back up, do a the update stuff I did way at the top of this post to make sure it's up to date. Yes I do the restarts again as I said above. Go ahead and yell at me...

Final Thoughts

Free stock photo from: https://www.pexels.com/photo/animal-cat-face-close-up-feline-416160/

You did it! You read this whole thing (it's a lot. So thanks). Here's a kitten picture and a couple of follow up thoughts.

More Clients, Fewer Clients, and Security Considerations

If you want more clients, just go way back up to the PKI section and follow the instructions for making client certificates with EasyRSA. You could/should generate client certs and keys on the actual client and only move the certificate signing requests (.csr files, remember?) to the CA. That's more secure. And the CA really should be on a separate server. That's more secure.

I chose not to do that because this is a cloud server which will be off most of the time, except when we want to use it. We expect it to get a new IP each time it boots up, and there isn't a reverse DNS lookup for it. We also locked down the ports with our security groups way back in Part 1, and we have the firewall on the server itself. I think it's more locked down than a lot of other possible implementations could be because of those factors. Finally, it's an EC2 instance, and so, ultimately, if we think it's been compromised we can just terminate it and start from scratch.

Certificate Revocation

On the topic of fewer clients, we can do certificate revocation for our clients if we suspect the keys have been compromised, or we let someone use our VPN and want to revoke that access.

Revoking a client is sort of easy, given all we've already done.

cd ~EasyRSA-3.0.4/
./easyrsa revoke badclient

Next we can make a certificate revocation list (CRL) and copy it to the right spot:

./easyrsa gen-crl
sudo cp pki/crl.pem /etc/openvpn

Once we put the CRL there we need to tell OpenVPN to actually use it. So open up the server config file at /etc/openvpn/server.conf and add at the bottom of the file this directive: crl-verify crl.pem.

Any time we do this to update the CRL, we need to update the OpenVPN server:

sudo systemctl restart openvpn@server

That's it. Thanks for reading this all. Now go outside, you've been inside for too long =D.