# Vault for Secret Management

Hello, Nice to see you again! 👋😊

🧵 [This is the part 05 of the Home Lab 2.0 series.](https://fewmorewords.com/series/home-lab)

Secret Management 🔐 is always a challenging exercise and like many of you, I have seen my fair share of credentials on plain text config files. Leaked and forgotten long living credentials in many shapes and forms are not a rare sight in todays IT system labyrinths. All the leading cloud providers have their own secret management solutions that tightly integrate with the rest of their eco-system. However they fall short in a multi-cloud environments.

**Hashicorp Vault** shines as an independent centralised highly capable secret management platform that can be integrated with many cloud providers as well as other existing solutions. Though I have used `Vault` in a few of my previous work places (including the current work setup) I didn’t have my own environment to tinker with as I wish. Therefore I decided to deploy my own instance of [Hashicorp Vault](https://www.hashicorp.com/en/products/vault) to manage secrets in my Home Lab setup. Of course, I will get a few more gray hair in the process but thats a small price to pay 🤩

## Host Selection 🏛️🏠

My initial plan was to use a VM to deploy Vault as a service or use a bunch of containers with `Docker Compose`. However during this Home Lab 2.0 journey, it was apparent that `LXC Containers` are perfect choice for most of my applications. They are super efficient (easy on my `poor mans` hardware 🫠) and first class citizens in a Proxmox setup. Though LXC containers don’t offer a similar level of isolation as a virtual machine, running them as `unprivileged` with a minimal up-to-date Linux destro template offers a balanced outcome for a Home Lab setup. Therefore I opted for an `Ubuntu based LXC Container` as my Vault host.

![Vault LXC Container](https://cdn.hashnode.com/res/hashnode/image/upload/v1756650663098/b7f6151c-81bf-43d2-a77c-08f7fc8e9c4c.png align="center")

I could easily reuse my `Terraform` templates that I used for `Keycloak` and `Traefik` deployments to get the host up and running. I have a separate `VLAN` for the Home Lab and that makes my life easier when it comes to allocate a static IP to the hosts.

<div data-node-type="callout">
<div data-node-type="callout-emoji">📣</div>
<div data-node-type="callout-text"><a target="_self" rel="noopener noreferrer nofollow" href="https://github.com/jayanath/home-lab/tree/main" style="pointer-events: none">By the way, the code for my Home Lab 2.0 is on Github and you will be able to easily adopt it for your own setup if needed.</a></div>
</div>

## Install and Configure Vault 🛠️🧑‍💻

This is the first time that I ever tried to setup my own `Vault`. Just running a Vault docker container in my Mac to poke around doesn’t really count, right?😊. Therefore, I took some time and did a bit of a digging to figure out the best way to set this up. Usual AI overlords `(Gemini, Claude, ChatGPT)` provided me some useful configuration help (most were riddled with errors) but I went through a lot of iterations to get things working with `Ansible`.

I decided to install `Vault` using `apt` as that was the most straightforward way to install Vault in Ubuntu. Since [Hashicorp recommends using Vault’s Integrated Storage](https://developer.hashicorp.com/vault/tutorials/raft/raft-storage) `(Raft)` for most use cases, I didn’t bother to mess with anything else. Also it was the simplest for my need. As you may have noticed, I separated host setup `(Terraform)` and application installation and configuration `(Ansible, Bash)` as two distinct actions for my setup. This reduced the complexity of the Home Lab 2.0 code. The `seal/unseal` process of Vault after each restart was a separate piece of challenge. After looking into the available options I ended up with `file based auto unseal (duct tape-ish) solution` as an acceptable workaround ¯\\\_(ツ)\_/¯ for my Home Lab 2.0 setup. This approach stores the single shard unseal key in a file but if anyone/anything is able to get to that far into my network, I’m cooked without a doubt and will have a lot more things to worry about 😂.

Maybe I will think about [auto-unseal Vault using AWS Secrets Manager](https://developer.hashicorp.com/vault/tutorials/auto-unseal/autounseal-aws-kms) in future, as an improvement to the current setup 🤞.

## Integration with Traefik and Keycloak 🔗🤝

The [main Ansible role](https://github.com/jayanath/home-lab/blob/main/vault/ansible/roles/vault/tasks/main.yml) uses two separate roles for the Vault init and Keycloak setup. Once this got executed, I dropped in a [new config file into dynamic folder of the Traefik](https://github.com/jayanath/home-lab/blob/main/traefik/ansible/roles/traefik/files/dynamic/vault.yml), added a `vault-headers` section into `middlewares` config and re-run the [Traefik playbook](https://github.com/jayanath/home-lab/blob/main/traefik/ansible/site.yml). Also I had to update my DNS records in `AdGuard` and `pfSense`. My initial setup seems to paid off as the integration effort with Traefik and Keycloak became a quick and easy process than I expected 😍.

![Keycloak client configuration](https://cdn.hashnode.com/res/hashnode/image/upload/v1756652065563/66031b0e-8595-4486-9973-310ffbbd94ff.png align="center")

```yaml
# traefik/ansible/roles/traefik/files/dynamic/vault.yml
# make sure to add the middlewares as well.
http:
  routers:
    to-vault:
      entryPoints:
        - websecure
      rule: "Host(`vault.jayforweb.com`)"
      service: vault
      tls:
        certResolver: dnsresolver
      middlewares:
        - vault-headers
        
  services:
    vault:
      loadBalancer:
        servers:
          - url: "http://192.168.193.80:8200"
        healthCheck:
          path: "/v1/sys/health"
          interval: "30s"
          timeout: "5s"
 
```

```yaml
# traefik/ansible/roles/traefik/files/dynamic/middlewares.yml
http:
  middlewares:
    redirect-to-https:
      redirectScheme:
        scheme: https
        permanent: true

    vault-headers:
      headers:
        customRequestHeaders:
          X-Forwarded-Proto: "https"
          X-Forwarded-Port: "443"
        customResponseHeaders:
          X-Frame-Options: "DENY"
          X-Content-Type-Options: "nosniff"
          Referrer-Policy: "strict-origin-when-cross-origin"
```

## Vault is Alive 🎉🍾

As I mentioned above, it took me many iterations to get everything working and I think there will be other challenges when I try to use the Vault with GitLab and the Kubernetes cluster. So far, I’m happy about how everything is coming into place. The Home Lab 2.0 is already serving me well as it gives me the opportunity to experiment with all these amazing tools, in my own terms.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1756657441317/20bf6025-e313-475f-b128-d6f3d5f3b8ac.gif align="center")

I sincerely hope that my effort in writing these things and sharing the not-so-perfect code will help at least one more person to figure out a thing or two.

Next, onwards to setup GitLab!

Happy tinkering! 😊

𓇢𓆸🧘🏻‍♂️𝓟𝓮𝓪𝓬𝓮

## Reference 📚📖

[Vault Installation - Official documentation](https://developer.hashicorp.com/vault/tutorials/day-one-raft/raft-deployment-guide)

[Vault Seal/Unseal process](https://developer.hashicorp.com/vault/docs/concepts/seal)

[Vault unseal options comparison](https://www.kloia.com/blog/comparison-of-unseal-options-in-hashicorp-vault)