Securely Accessing Private AWS Resources with TailScale

The Problem

One of the common problems of CI/CD automation is how to handle private resources without exposing these resources directly to the public or relying on complex VPN setups may not be the ideal solution.

Typically, you go with something like OpenVPN, but OpenVPN can be complicated, with something like OpenVPN,you have to set up your own server, and configure, maintain, etc. On the other hand, it takes less than 2 minutes to set up TailScale, Another thing, if you live in a country where OpenVPN is blocked, you don’t have many options.

Here is the diagram that we are going to be implanted in this blog, the blog post will guide you with step-by-step through its implementation. The initial steps involve setting up a GitHub Actions workflow that installs the TailScale client and authorizes it into a TailScale network. Subsequently, the blog post will cover the creation and configuration of the TailScale router, ensuring proper traffic routing and security group settings on AWS.

Connecting GitHub Actions to the Tailscale Network

Before starting this guide, you’ll need to have a Tailscale network set up and configured with at least one existing device, If you do not know how, follow this guide and of course you’re going to need an AWS account as well.

Now the next thing that I’m going to do is go out to GitHub and create a new repository,You can find the code for the end result in this repository.

You will then need to create an OAuth client to use with the Github action via this link, copy the Client ID and Client Secret to a safe place for now.

Go to the GitHub repo you created earlier and go to Settings > in the Security section of the sidebar, select Secrets and Variables, then click Actions, Create a GitHub secret with the name TS_OAUTH_CLIENT_ID and assign your OAuth client ID as the secret value and name TS_OAUTH_SECRET and assign your OAuth client secret as the secret value.

You will then need to create a server role account using ACL tags in Tailscale, basically ACL tags allow you to assign an identity to a device that is separate from human users, and use that identity as part of an ACL to restrict access. This should be used when adding servers to your Tailscale network, to learn more about ACL tags, click here

Open your favorite editor and create a .github directory and underneath .github we’ll create a Workflows directory, this is the special directory that GitHub Actions is going to look for workflow files in, create a new workflow, I’m going to call it tailscale.yaml.

And inside of this file, we’re going to go ahead and set up our basic workflow structure.

on:
  workflow_dispatch:
  push:
 
jobs:
  deploy-via-tailscale:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci
 

Let me break it down for you.

• the workflow dispatch event. Allows us to trigger workflows manually, which is great for debugging.
• The push event means that whenever we push a change, it goes ahead and triggers a build.
• Define the job and name it deploy-via-tailscale. It will run on the latest release of Ubuntu.
• connect to your tailnet by using the Tailscale GitHub Action

Go ahead and push the code to Github, you should see an ephemeral node. The node can access nodes in your Tailnet, subject to the access rules that apply to the specified ACL tag(s).

Set Up Amazon VPC and Tailscale Router

What we’re going to do is we’re going to create a VPC, and then we’re going to have this public subnet, and we’re going to put an EC2 instance on that public subnet, and we’re going to have that EC2 instance connect out to our Tailscale network as well, and then we’re going to set that EC2 instance up as a router.

Let’s go to the AWS Management Console. First, we’ll create an elastic IP and EC2 instance on the public subnet, which will act as a subnet router.

After the instance has passed the checks we need to download and install Tailscale onto your subnet router machine.

curl -fsSL https://tailscale.com/install.sh | sh

echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
sudo sysctl -p /etc/sysctl.d/99-tailscale.conf

sudo tailscale up --advertise-routes==10.0.16.0/20 --accept-dns=false

on:
  workflow_dispatch:
  push:
 
jobs:
  deploy-via-tailscale:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci
    - name: test-connectivity
      run: |
        ping -c 10 100.73.32.96

Create Private Amazon RDS Instance and Test Connectivity

After reviewing and verifying, we will provision an Amazon RDS instance in our VPC and test connectivity from our GitHub Actions workflow.

First, we will create a security group that will allow connection from the subnet router, I’m going to call it the database security group.

I will not cover how to create an RDS database here. I assume you have already created an RDS database, please refer to the AWS documentation if you don’t know how to create one.

If you want to test connectivity between your machine and the private Amazon RDS instance type the following command. This action allows you to connect to the MySQL DB instance using the MySQL client, Replace endpoint with the DB instance endpoint (DNS name) and replace admin with the master username you used. Enter the master password that you used when prompted for a password.

mysql -h endpoint -P 3306 -u admin -p

If everything is configured correctly, you should be able to see this massage.

We will return to the Github action workflow to test connectivity, we will create three secrets for the database endpoint, password, and username.

Update the workflow with this to test connectivity. We can now connect to RDS from any node in the Tailnet, using the same DNS name used inside AWS.

on:
  workflow_dispatch:
  push:
 
jobs:
  deploy-via-tailscale:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci
    - name: Test-Connectivity-to-AWS-RDS
      run: |
        mysql -h ${{ secrets.ENDPOINT }} -u ${{ secrets.USER }} -p"${{ secrets.PASSWORD }}" -e "show databases;"

Conclusion

In this blog, we’ve demonstrated how to create a secure and seamless connection between GitHub Actions workflows and private AWS resources. This approach not only improves the overall security of your CI/CD pipeline, but also ensures that sensitive data and interactions remain within your network, TailScale is easy to use, and has robust encryption mechanisms.