Software Engineering

Building a Kubernetes Cluster on AWS EKS using Terraform – Part V

Torsten Naumann
Torsten Naumann

Part V – creating the Application Load Balancer

In the last part of the series, we created and configured the EKS cluster, including both the master and a desired number of worker nodes. We can connect to it via kubectl, but we can’t yet access the cluster the way a normal user would do it: through a fixed URL.

We will now take a look at AWS Application Load Balancers. These are proxy instances that can be pointed at an instance or a group of instances in your AWS account. When the ALB is accessed, it forwards the request to these instances.

We use traefik to manage the service routing on our Kubernetes cluster; it works like a charm using the ALB as external Load Balancer.

Establishing the connection between an ALB and instances

A ALB provides a single endpoint for your cluster, but it needs to know what kind of instances to forward each request to. We add a Target Group to the definition of the worker nodes we created in the last article to solve that:

The „aws_lb_target_group“ is a new resource we create, which is then added to our „aws_autoscaling_group“. This makes it so that every instance created by out Autoscaling Group is automatically also a member of the Target Group we created. We can then reference the Target Group in our ALB definition.

Creating a managed Application Load Balancer

As always, we first need to create a fitting Security Group with rules allowing the resource to receive traffic:

Then we can go ahead and create the Application Load Balancer. This is relatively simple, we just need to reference our relevant subnets and the created Security Group. I also added the Security Group for our Kubernetes nodes to allow the Load Balancer to communicate with each node in a convenient way.

This is not entirely enough though – still need to create the rules by which our ALB forwards request. We do this in the form of Listeners:

We add one for unsecured HTTP via port 80 – this doesn’t forward any requests though, it just redirects them to the secure HTTPS listener, which we add second.

To make HTTPS properly work, we need to add a certificate to our Listener. Although one can be created with Terraform in theory, there isn’t really a good way to do so using AWS-based certificates.

UPDATE: I was made aware by @StGebert on Twitter that „aws_acm_certificate_validation“ can likely be used to make the certificate available directly in Terraform, which would probably be easier than setting it up before and using a data source like I did. Didn’t try it out yet, but I recommend taking a look!

We use a certificate we have to create before we run Terraform to create all of our resources. Then we import it with a very simple data source:

Note that we also added the previously create Target Group to the Listener – this way, all requests on port 443 using HTTPS will be forwarded to one of our worker nodes depending on the ALB configuration. This can be round-robin or based on the work load each node has to deal with at the time of the request.

With the way we set up our Target Group, all requests coming from the ALB will be forwarded to a worker node on port 31742. We can then configure a Kubernetes router listening to that node port on our cluster to properly route our traffic into our services – we use traefik for this.

Making our cluster endpoint human readable

With all that said and done, our cluster is available at the public endpoint for our ALB, which looks similar to this:

That is… not very easy to remember. We should add a DNS entry to fix that! And we can also do this using Terraform, by adding resources for the AWS Service Route53.

Depending on the rest of the contents of the AWS account we use, a public hosted zone might be already set up. While you can create this using Terraform too, the ACM certificate we need for the ALB Listener can be bound to a public hosted zone – and since we need the certificate before we can run our scripts, we have to assume at this point that the related public hosted zone has already been created, too. Therefore, we supply that hosted zone to our script using two parameter variables.

We can then create a Route53 record:

If you use a nodeport-based router in Kubernetes, a wildcard record is a good way to set up this DNS record –  any request ending in api.x will be redirected to the Kubernetes cluster while containing a header with the original request URL, so that the router in Kubernetes can use this to forward requests to the services you deploy.

Ready for service setup!

With all this said and done, you now have a running Kubernetes cluster on AWS EKS, which can be addressed from the public internet to access your services or via SSH to access the Kubernetes management and configuration. There are various other things you can set up using Terraform – we set up our databases in the database subnet using Terraform, for example.

These articles covered a lot of rather complicated AWS resources – if you have any questions or spot any mistakes in my guide, feel free to leave a comment or message me on Twitter at @_torstennaumann.

As always, you can check out the code in my GitHub-Repository for the article series. Don’t forget to enter your values for the access keys and region in the .tfvars file and the state bucket configuration before running it.