# EKS with ArgoCD using Terraform Recently I wanted to try out a few things with `EKS` and `ArgoCD`. I ended up writing a bit of Terraform to make it easier for me to create and destroy the infrastructure in my test AWS account. The code in [k8-eks-argocd-terraform](https://github.com/jayanath/k8-eks-argocd-terraform) deploys all theseπŸ‘‡ things. * VPC and the baseline networking * EKS cluster * AWS Load Balancer Controller * ArgoCD with Ingress * ArgoCD UI over πŸ” https with ACM cert * [**ArogoCD ApplicationSet**](https://argo-cd.readthedocs.io/en/stable/user-guide/application-set/) (this is cool!) * Sample applications under ApplicationSet I used `Kubernetes Manifest files` to deploy ArgoCD. This is because the existing [`Helm Chart` for ArgoCD](https://github.com/argoproj/argo-helm/tree/main/charts/argo-cd) doesn't support the new-ish **ApplicationSet** feature at the moment. I wanted to try this feature instead of using the good old `app of apps` workaround. I used the `git-dir-generator` and pointed it to the same repository to keep everything in one place. Rather than deploying from the laptop, I wanted to use `Github Actions` to deploy the infra πŸ€–. Though this made things a bit complicated, it was fun to experiment with Github Actions, and *reusable* workflows. I separated the code into `vpc`, `eks` and `argocd` for me to create/destroy parts of the infrastructure when needed. ## ♻️ Is This Code Reusable? Of course 😊. I tried to keep things as generic as possible but you still need to update a few things to get this working with your AWS account. ☠️ **WARNING!!** ☠️ **This is going to cost you a little. Please make sure to tear down everything once you are done with your testing.** Start with forking the repo [jayanath/k8-eks-argocd-terraform](https://github.com/jayanath/k8-eks-argocd-terraform). If you don't care about deploying the infra using Github Actions, just clone this repo instead. ### 1️⃣ Click-Ops bits As with any πŸ” and πŸ₯š problem, there are few `Click-Ops` bootstrapping bits. This is needed even if you want to deploy from your laptop. * Create an `S3 bucket` for you to store terraform remote state. (yeah, the local state works too, but..) * Create a `Hosted zone` in `R53` with a public domain * Create an `ACM cert` issued for `yourdomain.com` and `*.yourdomain.com` I just created the hosted zone and ACM using click-ops πŸ€·β€β™‚οΈ. Those cert validations take time and I don't have my domain registered with R53. It's with Google for safekeeping 🀫. ### 2️⃣ Github Actions bits Skip these steps if you are deploying from your laptop.
Create an `IAM role` with enough permissions for Github Actions ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667564743824/d2ozDTrYs.png align="left")
Create a Github secret `AWS_CI_ROLE` with the IAM role ARN ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667564849001/J3CjEMM3v.png align="left")
Configure `Open ID Connect (OIDC)` for Github Actions in AWS. Just follow the docs in the reference section. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667564949329/DD1iybKCj.png align="left")
### 3️⃣ Find/Replace bits This is to update the bucket name, domain name etc in a few places in the code before deployment. * `k8-eks-argo-terraform-state-jay` with your S3 πŸͺ£ name * `argo.jayforweb.com` with `argo.yourdomain.com` * `eksadmin` with your AWS IAM user if you want to use `kubectl` from your laptop * `us-west-2` with your region. ⚠️ Make sure to update `aws_image_repository` URL if you change the region. ### 4️⃣ Time to Deploy I'm sure you already know this but we need to follow the order of `vpc` --> `eks` --> `argocd` when deploying all this. If you are doing it from your laptop, just follow the drill of `terraform init`, `terraform plan` and `terraform apply` from within `vpc`, `eks` and `argocd` folders. If you set up the Github Actions, all you have to do is to select the action and run it. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667562662800/ivTlSEZs2.png align="left") EKS takes about 15-20 minutes to deploy. Maybe there is a chance to fix a β˜• until the machines in the cloud do their thing 😜. This is how long it took me to deploy everything. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667569101672/RHPlDHQ6N.png align="left") ### 5️⃣ Post Deployment Stuff 1. Add an alias record with the Ingress Load Balancer URL. \`AWS LoadBalancer Controller\` dynamically deploy a new LB or add new ingress into the same LB based on the setup. I couldn't figure out a way to automagically add this record to the hosted zone as I had no way of knowing when would the LB gets created. 2. Add alias records for sample apps, nginx1, nginx2 and nginx3 and target the same ALB. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1692533215187/c9362bbd-6e4e-4bdc-98f9-880a2d26aff4.png align="left") 3. Add your IAM user (\`eksadmin\` in my case) to the AWS configuration. Then update the \`kubeconfig\` to get access to your brand new EKS cluster and grab the ArgoCD default password from the \`argocd-initial-admin-secret\`. 1. \`export AWS\_DEFAULT\_PROFILE=eksadmin\` 2. \`aws sts get-caller-identity\` 3. \`aws eks update-kubeconfig --name eks-demo --region us-west-2 --profile eksadmin\` 4. \`kubectl get secrets -n argocd\` !\[image.png\](https://cdn.hashnode.com/res/hashnode/image/upload/v1667571895727/bxQU2HTnW.png align="left") 5. \`kubectl get secret argocd-initial-admin-secret -n argocd --template={{.data.password}} | base64 -d\` ### 6️⃣ The End Result πŸŽ‰ ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667573856669/ZKL0Cw5Lu.png align="left") ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667572376206/JS-ERyz8W.png align="left") ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1692533292728/4dbdbfae-9cb3-43eb-9dfa-c62a13e5da0c.png align="center") ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1692533401661/3a830b2a-48c0-4007-9f41-3efda011ccfb.png align="center") Now you could experiment by adding/removing/updating the sample applications. Once you commit the code, ArgoCD will faithfully adjust the apps in the cluster. ### πŸ”₯ Tear Down ☠️ **This is super important! If you leave this setup running for long time, it's going to cost you an 🦾 and a 🦿before you know it. Make sure to use** `terraform destroy` or those Github Workflows to nuke everything.
Sometimes, AWS fails to destroy certain resources.

The underlying issue is with the ENIs. This is a very old issue with AWS. Just go ahead and delete the ENI and security groups manually. image.png image.png image.png

![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1667576578358/AbN3tc8CU.png align="left") That's about it! If you reach this far, thank you πŸ™‡β€β™‚οΈ for your time. As always, just let me know if you got stuck, or if you have any questions. I'm more than happy to help! πŸ˜„πŸ‘. ### πŸ“š Reference 1. \[OIDC setup instructions for AWS\](https://docs.aws.amazon.com/IAM/latest/UserGuide/id\_roles\_providers\_create\_oidc.html#manage-oidc-provider-console) 2. \[OIDC setup instructions for Github\](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services) 3. \[ArgoCD ApplicationSet\](https://argo-cd.readthedocs.io/en/stable/user-guide/application-set/) 4. \[jayanath/k8-eks-argocd-terraform\](https://github.com/jayanath/k8-eks-argocd-terraform)