Traefik as the Reverse Proxy

Hey there 👋😊

🧵 This is the part 03 of the Home Lab 2.0 series.

Here, I’m going to discuss about my experience around deploying traefik as a Reverse Proxy to enable TLS for my Home Lab applications. No more browser warnings and everything got a nice little secure 🔐 URL.

Thought I setup the Proxmox and CEPH clusters manually, I’m using Terraform and Ansible to deploy and configure traefik.

Bootstrapping 🥾

There were a few steps that I had to take before beginning my deployment.

1. Setup SSH access to the Proxmox cluster ( yeah, I missed this step when setting up the Proxmox cluster 😅). This only takes few steps.

   1. Log into any of the Proxmox nodes and start a shell session.

   2. Add my public key to /etc/pve/priv/authorized_keys

   3. Uncomment PubkeyAuthentication yes on /etc/ssh/sshd_config

   4. Restart ssh daemon using systemctl restart ssh

2. Setup a new user in Proxmox to use with Telmate/proxmox provider. This process is clearly defined in the provider documentation. The documentation explains about how to create the API tokens as well.

3. Create a new S3 bucket to store my terraform state remotely. I created jay-home-lab-tf-state for this project with a minimum privilege IAM user.

4. export AWS_PROFILE, PM_API_TOKEN_ID and PM_API_TOKEN_SECRET

Deploy LXC with Terraform 🏗

Proxmox supports LXC containers and they are super resource efficient. LXC containers are sitting somewhere in the middle of Containers and full blown Virtual machines. They are quick to spin up and I decided to deploy traefik in a LXC container. There are many examples out there to explain how to deploy traefik v3.x as a docker container. However I could not find anything that clearly explains about deploying traefik v3.4 in a vm or a LXC container. Unfortunately the traefik docomentation didn’t offer much help either. In-fact the official documentation is more like a labyrinth😓 where you get lost without much hope.

After spending some time trying to figure out the new traefik v3.4 configurations, I stumbled upon this old video from Hussein Nasser. He explains about the traefik concepts and the basic configurations without fluff 🤗. Hussein’s youtube channel is a gold mine. Check it out and you will not regret it. Using that as a start, I was able to figure out the rest of the stuff without much trouble.

At first I created an LXC module but decided against it at the end as it is just a single resource. Also I wanted to use debian-12-standard_12.7-1_amd64.tar.zst but it was taking over 30 seconds to establish ssh connections. Something weired was going on with that and after restling with it for a few hours I opted for ubuntu-24.04-standard_24.04-2_amd64.tar.zst. That LXC image worked like a charm at the first go.

If you are interested, the Terraform code for the LXC container deployment can be found here.

Install and configure Traefik with Ansible 🧩

Once the LXC container is deployed I started working on the configuration part of traefik. I wanted to use my test domain jayforweb.com and obtain a wildcard cert from Lets Encrypt for the Home Lab applications. journalctl -u traefik -f was my best friend when trying to get the config files in order.

I used ansible-galaxy role init traefik to generate the scafolding for Ansible bits and it worked out quite well. If you’ve noticed, I’m using AWS Route53 to manage my DNS. Therefore I had to provide AWS credentials.

Few notable points ⚡

• It is best to use ansible-vault encrypt files/.aws/credentials to encrypt the aws credentials.

• Run the LXC container as Unprivileged and add grant the required capabilities. In this case traefik process requires CAP_NET_BIND_SERVICE capability to avoid port binding permission errors when serviving on port 80 and 443.

• Create an empty acme.json file with 0600 permission. This is done by the ansible playbook.

• Setup ssh access for Ansible. I’m using 1password to store my ssh keys. With that 1password ssh agent intercepts the ssh key requests and prompt me for the authentication. This works really well and I think its a great way to protect the ssh keys.

• Use ansible-playbook site.yml -i inventory.ini --ask-vault-pass to deploy the playbook. The —ask-vault-pass allows ansible to decrypt the credentials.

Since I first installed and configured traefik manually to understand the configuration caveats the automating everything with ansible was a relatively smooth process.

Below is the shiny new traefik dashboard. I will setup authentication to the dashboard when I deploy Keycloak and Vault.

Check out the code for the deployment of traefik. Hopefully it will be useful to you as well in case you want to deploy traefik in your own setup 👍🏻.

Also don’t hesitate to ping me for any questions related to the code. I’m more than happy to share my thoughts 😊.

See you next time with details about setting up Keycloak 👋🏻